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ABSTRACT 



"Advances in Education Research" reprints previously 
published journal articles reporting on research supported in whole or in 
part by the Office of Educational Research and Improvement (OERI) . The 
articles are selected from peer-reviewed/ref erred journals; the journals used 
are described briefy at the end of the volume. The articles in this volume 
are all concerned with reading comprehension and subject- area learning: (1) 

"The Contribution of Prior Knowledge and Coherent Text to Comprehension" 
(Margaret G. McKeown, Isabel L. Beck, Gale M. Sinatra, and Jane A. 

Loxterman) ,* (2) "Inclass or Pullout: Effects of Setting on the Remedial 

Reading Program" (Rita M . Bean, William W. Cooley, R. Tony Eichelberger , 

Meryl K. Lazar, and Naomi Zigmond) ; (3) "Individual Differences in 

Reprocessing of Text" (Dieter Haenggi and Charles A. Perfetti) ; (4) "The 

Process of Understanding: Reading for Literary and Informative Purposes" 
(Judith A. Langer) ; (5) "What Influences Learning? A Content Analysis of 

Review Literature" (Margaret C. Wang, Geneva D. Haertel, and Herbert J. 
Walberg) ; (6) "Revising Social Studies Text from a Text-Processing 

Perspective: Evidence of Improved Comprehensibility" (Isabel L. Beck, 

Margaret G. McKeown, Gale M. Sinatra, and Jane A Loxterman) ; and (7) "Sense 
Making and the Solution of Division Problems Involving Remainders: An 
Examination of Middle School Students' Solution Processes and Their 
Interpretations of Solutions" (Edward A. Silver, Lora J. Shapiro, and Adam 
Deutsch) . Each article contains references. (SLD) 
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Foreword' 



Advances in Education Research makes available to the public peer-reviewed, schol- 
arly research supported in whole or in part by the Office of Educational Research and 
Improvement (OERI) through its educational research and development programs. The 
goals of Advances in Education Research are to bring together from diverse scholarly 
sources first-rate, exemplary research that relates to important educational themes or 
topics; disseminate the results of funded research more widely to researchers, educa- 
tors, and policymakers; serve as a forum for discussing, debating, and exchanging 
research results and perspectives of researchers and education practitioners; and 
increase public awareness of, access to, and use of high-quality education research 
that is central and indispensable to improving and strengthening American education. 

This second volume of Advances in Education Research includes previously published 
articles from selected refereed journals, which are briefly described at the end of this 
volume. The articles are reproduced with the permission of the authors and the journals 
in which they originally appeared. They were written by individuals who received 
funding through OERI’s Field-Initiated Studies program for conducting research on 
how children learn to read and how they learn various academic subjects. 

Advances in Education Research is produced by the National Library of Education 
within OERI. The mission of the National Library of Education, as noted in Access 
For All, the report of the National Library of Education Advisory Task Force released 
in February 1997, is to: 

Provide awareness and maximum access to high-quality education 
information in all formats and from multiple sources, to provide high- 
quality assistance to information seekers, and to be accountable for all 
these efforts. Our customers are the educators, students, and parents of 
America at all education levels and the agencies, corporations, and 
institutions that serve them. Our partners are the nation ’s education 
libraries, education information providers, and education 
organizations. 

For more information about Advances in Education Research or the National Library of 
Education and the work it supports, contact staff at: 

The National Library of Education 

555 New Jersey Avenue NW 

Washington, DC 20208-5721 

Phone: 1-800-424-1616 or 202-219-1692 

E-mail: library@inet.ed.gov 

Web: http://www.ed.gov/NLE 
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This volume presents selected research articles related to the 
issues of literacy and subject-area learning. Each article in 
this volume was previously published in a refereed journal. 
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Introduction 



As the federal government’s main resource center for education information, the 
National Library of Education (NLE) provides three types of services— reference and 
referral, collection and technical, and resource sharing. In addition to housing micro- 
fiche catalogs and CD-ROM databases, NLE maintains a vast collection of printed 
resources, including more than 200,000 books and nearly 800 periodical subscriptions. 
NLE also offers specialized subject searches, electronic database search and retrieval, 
document delivery, bibliographic instruction, training services, interlibrary loans, 
legislative reference services, and selective information dissemination. 

In its role it as the nation’s collector and creator of education research information, 
NLE produces publications such as Advances in Education Research to promote re- 
cent research findings and disseminate the latest education information. This issue 
of Advances in Education Research presents seven articles that cover two themes: 
reading comprehension and subject-area learning. The articles unify these two 
themes under a central focus— the processes through which students understand, 
interpret, and use information. To investigate these processes, the researchers use 
a variety of approaches, including direct observation, case study analysis, experimen- 
tal intervention, and research synthesis. Regardless of the methods or the objectives 
of the individual studies, the researchers work toward a common goal— improving 
students’ abilities to process and use information. 



Reading Comprehension 



Because reading comprehension is a complex process, it has many unique aspects that 
can be studied from numerous perspectives. Interested in the aspect of influences on 
comprehension, McKeown, Beck, Sinatra, and Loxterman ( The Contribution of Prior 
Knowledge and Coherent Text to Comprehension) explore how background knowledge 
and text coherence affect reading comprehension. McKeown et al. begin their study 
with the premise that coherent text clearly improves reading comprehension, and they 
expand on this idea by examining the influences that background knowledge has on 
comprehension. 

For their study, McKeown et al. separate 48 fifth-grade social studies students into two 
groups— those who read an original text and those who read a revised version of that 
text. Before reading the texts, the students are taught a knowledge module that provides 
background information not presented in the texts. After reading the texts, the students 
are questioned to determine the level of their comprehension. 

The initial findings of the study only corroborate conclusions of other studies that show 
a clear correlation between coherent text and improved reading comprehension. To 
extrapolate additional meaning from their findings, McKeown et al. compare these 
findings with results from an earlier study that focused only on the influence of co- 
herent text. (They justify comparing both sets of data because the same experimenter 
gathered the data from the same school using the same curriculum and the same fifth- 
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grade social studies teacher. The only difference is that in the earlier study, the students 
did not receive the background information.) 

After comparing the data from both studies, McKeown et al. discover that the inclusion 
of background knowledge is “not an overall general effect” and that text coherence 
“seems to be the stronger enhancement for understanding complex content. ” Despite 
this conclusion, the combination of background knowledge and coherent text appear to 
provide “an advantage beyond the sum of what the two provide separately.” 

During the course of their study, McKeown et al. unintentionally experience another 
phenomenon associated with background knowledge. This phenomenon occurs when 
background knowledge overrides text information “in such a way that readers recall 
text or respond to questions with information from their knowledge base, even though 
it is not supported by— and in some cases is even contradictory to— the text.” Although 
this phenomenon is prominent in current literature, it requires further exploration. 



Studying a different aspect of reading comprehension, Bean, Cooley, Eichelberger, 
Lazar, and Zigmond ( Inclass or Pullout: Effects of Setting on the Remedial Reading 
Program) examine how classroom settings affect Chapter I remedial reading programs, 
focusing on the pullout and inclass models. In the pullout model, students leave their 
regular classroom setting to receive instruction from a reading specialist, while in the 
inclass model, the reading specialist enters the regular classroom to determine student 
needs and work with specific students. 

Using data from a larger study, Bean et al. are mainly concerned with the nature of 
each type of remedial reading program and with differences in instructional behaviors, 
lessons, and reading behaviors. To help control for reading philosophy, materials, 
community characteristics, and teacher quality, they gather data from a school district 
containing several schools that implement each type of program. Relying on classroom 
observations, Bean et al. analyze how time is allocated in each program, concentrating 
on how reading specialists teach, how students behave, what lessons are taught, and 
what activities are completed. 

Their findings uncover a complex combination of similarities and differences that 
“seem to have an effect on the nature of the instruction experienced by students.” 
Among the similarities, students in both programs receive instruction as members of 
a group, they spend minimal time pursuing independent activities such as workbook 
or worksheet assignments, and they spend most of their time listening. Among the 
differences, pullout programs focus mainly on skill instruction while inclass programs 
emphasize study skills or testing activities, and students in pullout programs spend less 
time in transition while those in inclass programs have more time without interaction or 
scheduled instruction. 

Although Bean et al. conclude that “setting can make a difference in terms of what 
students experience in compensatory education,” they admit that changing settings 
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cannot adequately integrate students into regular education programs because switching 
from a pullout to an inclass program “appears to be much more complex and perhaps 
less effective than one would hope.” For this reason, they suggest that additional re- 
search must be completed to determine the effectiveness of integration and alternative 
models for compensatory programs. 



Concentrating on a more specific area of reading comprehension, Haenggi and Perfetti 
( Individual Differences in Reprocessing of Text) examine how basic reading processes 
and prior knowledge affect text reprocessing. Haenggi and Perfetti begin their study by 
separating a class of 48 undergraduate students into two groups according to reading 
ability— average and above-average readers— and further subdividing each group 
according to three reprocessing conditions— rewriting notes, rereading notes, and 
rereading text. After establishing these groups, they test the students’ comprehension 
of three types of information— explicit, text-implicit, and script-implicit— and use these 
information types to determine the influence of working memory and prior knowledge. 

Because of the numerous variables explored in their study, Haenggi and Perfetti are 
able to make several conclusions. Most notably, they determine that reprocessing text 
generally improves reading comprehension, regardless of the reprocessing condition, 
and that “individual differences in basic reading processes and prior knowledge con- 
tributed more to reading comprehension than did the specific reprocessing activity. 
Using additional findings, they conclude that reprocessing affects average and above- 
average readers differently. 

Despite Haenggi and Perfetti’s exploration of multiple variables, several hypotheses 
remain untested. For example, less-skilled readers may find rereading notes more 
helpful in improving their comprehension than other strategies, while skilled readers 
are less affected by reprocessing strategies. Also, the availability of retrieval cues may 
“be the reason for the weak relation between the number of noted idea units and 
comprehension accuracy after a first learning trial ... as well as after reprocessing. 



Interested in the aspect of meaning making, the process readers use to develop an 
understanding of what they read, Langer (The Process of Understanding: Reading for 
Literary and Informative Purposes ) examines how middle and high school students 
create meanings as they read literary and informative texts. Focusing on literary 
meaning making, Langer explores how students understand literature and how the 
development of this understanding relates to informative reading selections such as 
science and social studies. 

Using think-aloud sessions to analyze the meaning-making process, Langer determines 
that reading “involves a variety of changing stances that the reader takes toward the 
text.” These stances are recursive, and readers experience them while reading either 
literary or informative texts. The four stances Langer defines are (1) being out and 
stepping into an envisionment, (2) being in and moving through an envisionment, 
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(3) stepping back and rethinking what is known, and (4) stepping out and objectifying 
the experience. 

Lunger also discovers that in addition to experiencing local envisionments of the text, 
readers assume different orientations in literary and informative contexts. In the literary 
context, readers sense of the whole continually changes and develops “as a constantly 
moving horizon of possibilities. ” In the informative context, the sense of the whole 
remains static as “a steady point of reference . ” These two orientations ultimately 
influence the readers concerns within each of the four stances and their approach 
to the meaning-making process. 

Although Langer’s research focuses on middle and high school students, the results 
have several broad applications. The concept of stances that reflect readers’ growing 
envisionments can offer an alternative to existing category systems that are primarily 
based on types or levels of response.” Furthermore, because these stances are linked to 
extending the understanding of texts, they can be used “to analyze the extent to which 
readers are or are not willing or able to extend their envisionments in order to reach a 
fuller response.” 



Subject-Area Learning 

Numerous variables influence students’ ability to learn any subject matter. Interested in 
those variables, Wang, Haertel, and Walberg {What Influences Learning? A Content 
Analysis of Review Literature) analyze 179 sources, including handbook chapters, 
annual review chapters, government documents, commissioned papers, book chapters, 
and journal review articles. Using these sources, they conduct a research synthesis to 
identify the variables that have the strongest influence on learning. 

Before beginning their research synthesis, Wang et al. identify 228 comprehensive 
variables that influence learning outcomes and organize those items into a conceptual 
framework based on the major theoretical models of school learning. This conceptual 
framework separates the variables into 6 categories that encompass 30 a priori scales. 
Wang et al. rate the 228 variables appearing in the source documents, using a detailed 
15-page coding form. After rating the variables, they analyze the coding forms “to 
determine the emergent consensus on which items exert the most powerful influence 
on learning outcomes.” 

This research synthesis confirms that quantity and quality of instruction, home environ- 
ment, and student characteristics are most important to learning outcomes because these 
items are “directly tied to students’ engagement with the materials to be learned.” Of 
the 30 a priori scales, the items having the strongest influence on learning are student 
metacognition, effective classroom management, quantity of instruction, positive and 
productive student-teacher interactions, a classroom climate conducive to learning, and 
a peer culture supportive of academic achievement. 
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Wang et al. note that many authors of research studies and reviews are often cautious 
when determining the influence that individual variables have over learning. However, 
this caution is very appropriate because this research synthesis confirms “that a large 
number of variables are moderately related to learning outcomes, but few, if any, 
single variables are strongly related to learning.” 



Concerned with the quality of instruction, Beck, McKeown, Sinatra, and Loxterman 
(Revising Social Studies Text from a Text-Processing Perspective: Evidence of Improved 
Comprehensibility) examine how improving the quality of textbooks can improve 
learning. Beginning with the premise that incomprehensible text inhibits the ability 
of readers to learn, Beck et al. revise passages from a fifth-grade social studies text- 
book so that the text relates information clearly. In addition to presenting the rewritten 
passages, they explain their rationale for the revisions to illustrate one process that can 
be used to revise textbooks. 

The goal of the revisions is “to create a text that would assist a reader in connecting 
pieces of text information and combining that information with knowledge to develop 
a coherent text representation.” By revising text to activate relevant schema, add 
explicit signals, and remove extraneous information, Beck et al. ensure that causes 
are connected to events and that events are connected to consequences. After making 
appropriate revisions, they test fourth- and fifth-grade students to determine how 
revisions influence learning. 

The findings of their study clearly show that revising text for clarity improves students’ 
ability to comprehend and recall information. Furthermore, because the passages are 
revised to reveal cause-and-effect relationships among events, students are able “to 
explain the actions of the text and to move the chain of events forward.” 

Although their study demonstrates the need for quality textbooks, Beck et al. note two 
weaknesses to revising texts. First, simple revisions to text do not adequately explain 
complex issues because “even a coherent, well-designed text is unlikely to communi- 
cate to a young learner all the information that a mature reader could glean from it.” 
Second, “creating comprehensible text is a very complex task, and its complexity 
precludes devising a set of formulaic procedures.” Because of these weaknesses, they 
hope that the models accompanying their study “can serve as mechanisms for 
transferring knowledge to new instances.” 



Interested in students’ interpretations of numerical answers obtained from story 
problems, Silver, Shapiro, and Deutsch (Sense Making and the Solution of Division 
Problems Involving Remainders: An Examination of Middle School Students’ Solution 
Processes and Their Interpretations of Solutions) explore how mathematics students 
handle augmented-quotient division-with-remainder problems. Testing referential 
mapping models proposed in earlier research, Silver et al. examine students’ solution 
processes and their situation-based reasoning as they solve story problems. 
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To investigate these solution processes and the associated reasoning, Silver et al. give 
students a free-response pencil-and-paper task that requires students to calculate a 
numerical figure using a mathematic procedure and interpret that figure for the solu- 
tion to the story problem. In addition to providing the solution, students must explain 
the logic that they used to determine their answers. 

After analyzing their data, Silver et al. conclude that “computational requirements 
were not the major barrier to obtaining a correct solution but rather that unsuccessful 
solutions were more often due to students’ failure to engage in interpreting their 
computational results.” In another finding, they conclude that many students are more 
concerned about the form of their computations and final answers than “about the 
relationship between the numerical answer and the problem being solved.” Further- 
more, anecdotal evidence suggests that the written explanations of the solution reveal 
only a portion of the students’ situation-based reasoning. 

Because of these findings, Silver et al. believe that further research and improved 
instruction are necessary. In addition to promoting the investigation of the relationship 
between problem-solving procedures and situational contexts, they suggest examining 
“alternative assessment settings or formats that might target more directly the need for 
sense making and solution interpretation.” Finally, they recommend “developing and 
implementing instructional activities in which children are challenged to engage in 
mathematics sense making” to improve “the poor mathematics performance of 
American students.” 



In addition to providing a better understanding of the processes through which students 
understand, interpret, and use information, the seven articles contained in this issue 
present ideas and suggestions that can help improve students’ abilities to process and 
use information. These articles demonstrate the complex nature of learning, illustrating 
the importance of current research while presenting possibilities and directions for 
future research. The ultimate value of these articles is that they highlight the role of 
research in improving learning and the entire educational system. 
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Jane A. Loxterman 

University of Pittsburgh 



The contribution of prior knowledge 
and coherent text to comprehension 



L earning, both in school and beyond, is heavily 
dependent on acquiring information from text. 
Research on text processing over the last two decades 
has greatly expanded understanding of what is involved 
in reading and learning from text. The current view of 
reading has shifted from that of a simple process of 
lifting the message from a text to that of an active, com- 
plex process in which a reader draws on information 
from several sources concurrently to construct a repre- 
sentation of a text’s message. 

The constructivist orientation of recent research 
has highlighted the role of a reader’s background 
knowledge. Research has emphasized that not only 
does lack of knowledge about a topic impede compre- 
hension, but the extent of knowledge influences the 
quality of understanding that a reader can construct. 
Research by Voss and his colleagues (Chiesi, Spilich, & 
Voss, 1979; Means & Voss, 1985; Spilich, Vesonder, 
Chiesi, & Voss, 1979) and Chi and her colleagues (Chi, 
1978; Chi, Feltovich, & Glaser, 1981; Chi, Glaser, & 
Reese, 1982; Chi & Koeske, 1983) has shown the ad- 
vantage in comprehension for high knowledge versus 
low knowledge individuals. 

Another influence on a reader’s ability to acquire 
information from text is the way the text content is 
organized and explained. The concept of coherence has 
been used to describe the kind of organization of text 
that facilitates the reader’s task. Thus, coherence is the 
extent to which the sequencing of ideas in a text makes 
sense and the extent to which the language used to 
present those ideas makes the nature of the ideas and 
their relationships apparent. 



Research on text has provided evidence of thercia-. 
tionship between coherence and the comprehensibility 
of a text. Studies that present readers with more and 
less coherent versions of text have shown that the more 
coherent versions yield better comprehension (Beck, 
McKeown, Omanson, & Pople, 1984; Beck, McKeown, 
Sinatra, & Loxterman, 1991; Britton, 1990). The con- 
cept of coherence and its role in text comprehensibility 
is also an as pea of Anderson and Armbrustcr’s (1984) 
notion of "considerate text.” 

Research context for studying the contribution 
of knowledge and coherent text 

Advances in understanding the complexity of the 
reading process, particularly the role of the reader’s 
knowledge and the characteristics of text that influence 
comprehension, provide a rich context for investigating 
instructional issues. We have used this context as the 
base for a program of research on learning history from 
social studies textbooks. This research effort started , 
with an analysis of textbooks (Beck, McKeown, & 
Gromoll, 1989), followed by an investigation of stu- 
dents’ prior knowledge (McKeown & Beck, 1990), and 
then a study of the effects of more and less coherent 
texts on students’ comprehension (Beck et al., 1991). 
The present study, which looks at the effea of back- 
ground knowledge on more and less coherent texts, 
represents a culmination of this program of research. 

Analysis of textbooks. Beck et al. (1989), in an 
analysis of content and content presentation in four 
widely used social studies programs, concluded that the 
presentation of history in the programs, examined was 
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not oriented toward helping young learners develop a 
coherent representation of various historical topics and 
periods. At a general level, there seemed to be two re- 
lated reasons why this was so. First, the texts assumed 
an unrealistic variety and depth of prior knowledge 
from target-age students. Second, the presentation of 
content was less than coherent. That is, the texts tended 
to present numerous facts, but there was little explana- 
tion of relationships among facts. Such texts require 
young learners to make many connecting inferences 
among events and ideas with little, if any, assistance. 

Both issues — the unrealistic assumption of prior 
knowledge and the influence of textual coherence — 
were empirically examined in subsequent studies (Beck 
et al., 1991; McKeown & Beck, 1990). In both studies 
the topic selected for investigation was the period lead- 
ing to the American Revolution, which traditionally is 
first encountered formally in the fifth-grade curriculum. 
This topic was selected for study because of its impor- 
tance. That is, because the period embodies the princi- 
ples upon which the U.S. is founded, its influence is 
enormous for young Americans* subsequent learning in 
history, as well as other subject matters, and is strongly 
reflected in diverse aspects of American culture. 

Investigation of prior knowledge. The McKeown 
and Beck (1990) study examined fifth graders’ knowl- 
edge of the Revolutionary period to evaluate whether 
students had acquired the knowledge that was assumed 
by the text. Although in the Beck et al. (1989) textbook 
analysis the investigators judged that the passages on 
the Revolutionary War assumed a good deal of knowl- 
edge, it is possible that students have accumulated that 
knowledge before instruction from such sources as 
reading, television, and general references to the period 
that exist in American society. Thus, to probe the extent 
of students’ relevant knowledge, McKeown and Beck 
(1990) interviewed fifth graders just before they initially 
studied the Revolutionary period in school. The inter- 
view questions were based on two broad notions that 
Beck et al. (1989) indicated were largely assumed by the 
textbooks examined in their analysis and which they 
suggested are necessary for learning about the period: 
the role of England in the colonists’ struggle for inde- 
pendence and ideas about representative government. 

Students’ responses to the interview questions 
were used to characterize the knowledge that target 
learners bring to texts about the Revolutionary period 
and to assess the match between what students know 
and what the texts assume. The results suggested that, 
although many students had some familiarity with 
events and issues related to the period, for most stu- 
dents this knowledge was vague and often contained 
inaccuracies. Thus, to a large extent, the results sup- 



ported the hypothesis that knowledge assumed by the 
texts was not well represented in students’ repertoires. 

Comparison of more and less coherent text. The 
second problematic issue identified in the Beck et al. 
(1989) analysis was the less than coherent nature of the 
textual material. This problem was investigated by de- 
veloping revised versions of textbook passages and 
examining students’ comprehension of revised and 
original texts (Beck et al., 1991). The revised versions of 
text were developed within a cognitive processing ori- 
entation that involved simulating the process of a read- 
er’s interaction with a text. This simulation took into 
account a range of findings from text processing re- 
search about what causes reader/text interactions to 
proceed or to break down (see, for example, Graesscr & 
Clark, 1985; Just & Carpenter; 1987; Kieras, 1985; 
Perfetti, 1985; Trabasso, Secco, & van den Brock, 

1984). The goal of the revisions was to create a text that 
would assist the reader in connecting pieces of text 
information and combining this information with prior 
knowledge to develop a coherent representation. To- 
ward this goal, the revised text was based on a causal 
sequence of events with the information presented in 
such a way as to expose the reasoning that connects a 
cause to an event and an event to a consequence. 

At a general level, the revisions were intended to 
establish textual coherence by clarifying, elaborating, 
explaining, and motivating important information, and 
making relationships explicit. Because decisions under- 
lying the revisions were so strongly tied to the specific 
content we present a summary of the changes in terms 
of the content. (A more complete description of the 
revisions appears in Beck et al. , 1 99 1 .) The text we used 
comprised four segments: “French and Indian Wax;’’ 

“No luxation Without Representation,” “Boston Tea 
Party,” and “Intolerable Acts.” 

The original text on the French and Indian War 
failed to provide explanations of the motivation and 
consequences of the’ war between Britain and France 
and failed to make explicit the colonies’ relationship to 
Britain. The goal of the revisions for this segment was 
to present an explicit framework for the war (who 
fought and why, what resulted) and lay the groundwork 
that would provide a setting for conflict between Brit- 
ain and the colonies. 

The original “No luxation Without Representa- 
tion” text segment failed to communicate the cause of 
the colonists’ distress over taxes, in large part because it 
did not explain the concept of representation. The re- 
vised text explained that British citizens elected people 
to Parliament but the colonists could not vote in those 
elections and that this was the basis for the colonists! 
anger at being taxed by Parliament. 
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The original text about the Boston Tea Party, al- 
though it gave an adequate portrayal of the event itself, 
failed to explicate its role as a protest over Britain’s 
taxes. The revised text explained that the colonists’ 
anger over paying for tea stemmed from the tax on the 
tea. This anger about the tax is what led to colonial 
protests. 

Finally, the original text on the Intolerable Acts did 
not adequately portray the desperate situation created 
in Boston by Britain’s laws that were intended to punish 
colonists for the tea party. The revised text clearly de- 
tailed those punishments and explained the conse- 
quences for colonial life in Boston. The revisions were 
intended to help readers understand the causes of con- 
flict between Britain and the colonies and to prepare 
students to understand where the intensifying conflict 
was leading. 

The students who read the revised text showed 
significantly better comprehension than those who read 
the original text, and this included greater awareness of 
the role of central events in the causal sequence. How- 
ever both groups’ performance indicated that they still 
had considerable difficulty in understanding the text. 
One possible hypothesis for the comprehension diffi- 
culty is the lack of background knowledge identified by 
McKeown and Beck (1990). It is reasonable to assume 
that the background knowledge problems identified in 
McKeown and Beck were applicable to the population 
used in the Beck et al. (1991) study because the subjects 
in both studies were from the same small, stable school 
district with a traditional and consistent social studies 
curriculum. 



The present study 

Given that we had, in these previous studies, exam- 
ined students’ prior knowledge of information related 
to the text and devised versions of the text passages that 
improved coherence, the next logical step was to exam- 
ine how students would perform on the original and 
revised versions of the text if their background knowl- 
edge were upgraded. Thus the purpose of the present 
study was to provide students with relevant back- 
ground knowledge and test the effects of this knowl- 
edge on comprehension of more and less coherent 
versions of a text. The nature of the relationship be- 
tween knowledge and text structure— of which coher- 
ence can be considered an aspect — was the focus of a 
recent review by Roller (1990). In discussing conflicting 
findings about the role of knowledge and structure in 
text comprehension, Roller suggested that the influence 
of structure varies depending on the familiarity of the 



text content; in particular, structure is most facilitative 
when the content is moderately unfamiliar and de- 
creases in importance if readers have sufficient familiar- 
ity. In this study, then, the focus of interest was whether 
providing background knowledge would make the con- 
tent familiar enough to preclude a strong influence for 
the structural support provided by enhanced textual 
coherence. 

In the present study, background knowledge was 
provided to all students through a carefully crafted ex- 
perimenter-led presentation. Students were then as- 
signed to one of two text conditions: the original text 
materials from a fifth-grade social studies textbook or 
the revised version of the. text. The question posed in 
the present study was whether providing relevant back- 
ground knowledge before reading would compensate 
for the less coherent text, or whether an advantage 
would still be demonstrated for the coherent text. 



Method 

Subjects 

Subjects for this study were 48 fifth graders from 
an elementary school in a middle class small public 
school district in a northern U.S. state. Parental permis- 
sion was obtained for all students who participated. 
Subjects were rank ordered based on their reading com- 
prehension test scores on the Metropolitan Achievement 
Test. Every other student in the distribution of reading 
comprehension scores was then assigned to one of the 
two groups for the text conditions. This resulted in 
comparable means and standard deviations of compre- 
hension scores for the two groups (original group, 
66.46, SD » 23-52; revised group, 66.42, SD - 23.62). 

Materials and procedures 

All students received a prepared instructional mod- 
ule designed to provide relevant background knowl- 
edge for understanding the text. One of the authors 
presented the knowledge module, using a written 
script, to students in their regular social studies classes. 
An equal number of students in each of the two social 
studies classes was assigned to each text condition. 

Rationale for knowledge module . The content 
that was presented focused on establishing and distin- 
guishing the major agents of the Revolutionary period: 
the British and the colonists. This content was selected 
because of its role as background for the sequence of 
events to be presented in the text and because of evi- 
dence that such information would not typically be in 
students’ repertoires. 

The content of the knowledge module identified 
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the colonies as belonging to Britain; portrayed the colo- 
nists as beginning to develop an identity of their own. 
separate from being British; and explained that the colo- 
nists, although under British rule, were allowed to 
make some of their own laws. Each of these three con- 
cepts was included to provide students with an under- 
standing of significant features of the political and 
social context that existed prior to the Revolution. That 
is, it seems that if students failed to understand that the 
colonies were British, they would have little basis to 
understand what the colonies might want freedom 
from; further, the emergence of a distinct “American” 
identity is an important feature that motivates the desire 
for independence; and finally, some familiarity with 
governmental rights of the colonies is needed to under- 
stand why they might reject taxation without represen- 
tation and what denial of self-government under the 
Intolerable Acts would mean. 

Evidence that students might lack understanding of 
the selected content comes from analysis of the text- 
books (Beck et al., 1989) and from an empirical study 
of students' knowledge about the Revolutionary con- 
text, agents, and events (McKeown & Beck, 1990). 

From the analysis of textbooks, we found that Britain’s 
possession of the colonies was not strongly established 
in the text. The concept that colonists were beginning 
to develop an identity as Americans was almost com- 
pletely lacking, and colonial self-government was barely 
given mention. From the study of relevant knowledge 
of fifth graders just before instruction on the Revolu- 
tionary period, we found that only about 10% of the 
students related the 13 colonies to Britain, about a third 
of the students named Britain as a participant in the 
Revolutionary War; and many students lacked under- 
standing that the colonies had some powers of self- 
government. 

Presentation of the knowledge module. To estab- 
lish the purpose for the lesson, students were told that 
because they would soon be reading and learning about 
the American Revolution, which was described as “a 
war that the American colonists fought for their inde- 
pendence against Britain over two hundred years ago,” 
they were going to talk about some things that might 
help them understand that time in history; specifically, 
they were going to talk about some of the people and 
places that were involved in this period. 

The module began by establishing five geographic 
or political entities— North America, Britain, Fiance, 
Massachusetts, and Boston— that are referred to in the 
text and about which students evidenced confusion 
(Beck et al., 1991). A component of the instruction 
aimed at introducing these entities and relating them to 
what students might know about their existence today. 



To accomplish this purpose, some map work was done, 
and then the experimenter introduced two side-by-side 
charts. One was titled “Today, about 200 years after the 
Revolutionary War;" the other was titled “Before the 
Revolutionary War, between about 1760 and 1775.“ 
Under each title, the five entities were printed down 
the side of the charts. The charts contained descriptors 
for each of the entities. These descriptors were con- 
cealed when the chart was first introduced. 

Through a scries of questions, the experimenter 
established the entities in terms of what they are today 
and what they were before the Revolutionary War, un- 
covering each descriptor as it was discussed. For exam- 
ple, on the “Tbday" chan, Britain was described as “A 
country 3,000 miles away from North America. Some- 
times Britain is called England," and on the “Before the 
Revolutionary War Chan” as “The country that owned 
. the 13 original colonies.” When all the descriptors were 
uncovered, students were encouraged to engage with 
the information by using it to answer questions such as 
“Were the 13 original colonies near Britain?" (The rele- 
vant information on the charts was that the 1 3 original 
colonies were in North America and that Britain is 
3,000 miles away from North America.) 

The first portion of the module ended with a sum- 
mary emphasizing that the 1 3 colonics were owned by 
the British, and that in fact, before the Revolutionary 
Wax; the 13 colonies were considered part of Britain, 
even though they were far away. Students were told that 
the discussion would now turn to “the people who 
lived in the 13 colonics, the people we call colonists." 
The intent of this section was to develop the notion 
that over the years the colonists were losing their Brit- 
ish identity and developing an American identity. 

lb introduce this idea, the experimenter told the 
students that many of the people who came to the colo- 
nies in the 1600s were British. And even though they 
lived far away from Britain, they felt British. "Another 
way of saying that is: the colonists had a British iden- 
tity. Someone's identity is what they feel they are. You 
have an American identity because you feel American.” 
The experimenter explained that* as years went by, a lot 
of the colonists began to lose their British identity; they 
began to feel separate from the British, and even began 
to call themselves Americans. 

lb encourage students to engage with the informa- 
tion, the experimenter asked, “Why do you think there 
was this change?” and provided prompts and questions 
toward developing the notion that the colonists had 
established a very different life style from that in Britain 
(“Let's think of what the colonists had to do when they 
first came over here. Did they have houses to move 
into? Were there stores to go to?*'). Attention was also 
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brought to the influence of the passage of time in 
changing the colonists’ identity (e.g., "Let me tell you 
that a lot of time passed— about a hundred years— be- 
tween when people started coming to the colonies and 
the middle 1700s, the time we are talking about now. 
Why would that help bring about changes in how the 
colonists felt?”). 

In the final portion of the identity section, the 
experimenter read descriptions of people, and the stu- 
dents decided whether the individual would be more 
likely to have a British or American identity. For exam- 
ple: 

My name is Samantha Stevens. I spent many years of my 
life getting a school going in our town and helping to 
teach the children. I am very proud of the work I’ve 
done and of how much our children learn. In Britain, 
only those who can afford it send their children to 
school. That used to seem fine to me. But here everyone 
goes to school— and I really think that is the way it 
should be! 

What is Samantha s identity? 

Next, students were told that having thought a 
little about the people who lived in the colonies, they 
were now going to think a little about how the 13 colo- 
nies were governed. To start this section the experi- 
menter posed the question, "Who decided how things 
were run in the 13 colonies— Was it the British, or was 
it the colonists, or was it both — the British and the col- 
onists?” After the students made their choice, the ex- 
perimenter established that both the British and the 
colonists were involved in running the colonies. 

The experimenter explained that the 1 3 colonies 
“weren’t united into one country. Each of the 13 colo- 
nies had their own little government” and that an im- 
portant part of that government was something called a 
colonial assembly that made many of the rules and laws 
for the colony. It was also established that people be- 
came assemblymen through elections. 

Students were then told that “in addition to the 
laws the assemblies made, the government over in Brit- 
ain could make rules and laws that all the colonies had 
to follow.... So the colonies had some self-government, 
but they did not have complete self-government.” The 
final governmental information presented to the stu- 
dents was about Britain having a king and parliament, 
which was where the laws were made, "something like 
our Congress.” 

The last activity, intended to further reinforce the 
dis tinctions between the colonists and the British, in- 
volved "thinking a little more about the different ways 
that the British and the colonists acted and thought.” 
Here the experimenter introduced a chart that had on 



one side an illustration of some British characters, in- 
cluding King George, and on the other side some colo- 
nial characters. She then read some “quotes” that were 
explained as “something that someone in one of these 
groups might say,” and the students were to decide 
whether it was probably said by someone in the British 
group or someone in the colonial group. For example, 

“I vote for people who go to the Massachusetts Assem- 
bly.” “I can make laws for all the colonies.” "My grand- 
parents sailed 3,000 miles across the Atlantic Ocean.” 

“1 vote for people in Parliament.” 

In summary, the content in the 35-minute knowl- 
edge module was based on information identified as 
important background information that had been as- 
sumed by the textbook presentation. It is important to 
emphasize that the material in the instructional module 
did not present the sequence of events described in the 
text, but introduced major agents and prerequisite con- 
cepts needed to construct a representation of that se- 
quence from the text material. 

Administration of text conditions . Students were 
presented the text materials in individual sessions with 
an examiner. The examiner followed a written script 
that included a brief introduction, instructions, recall 
prompts, and short-answer questions. Each student 
read the text silently in four sections, one section at a 
time (i.e., “French and Indian War,” “No Taxation With- 
out Representation,” “Boston Tea Party,” and “Intolera- 
ble Acts”). Students were asked to think of the sections 
as passages from their social studies book. When stu- 
dents completed each of the four sections, they were 
asked to tell in their own words what the section was 
all about. Then students were asked a scries of open- 
ended questions. The recalls and answers to questions 
were tape recorded and later transcribed for scoring. 

Dependent measures and scoring 

The dependent measures and scoring schemes used 
in this study were the same as those used in Beck et al. 
(1991). 

Dependent measures. Measures of recall and ques- 
tions were used to assess students’ comprehension of 
the texts. Because young students’ recall protocols of 
unfamiliar content tend to be rather limited, an exten- 
sive set of questions was developed for each passage. 
The questions, which were open-ended in format, fo- 
cused on the sequence of events and the relationships 
among the people and events described in the text. For 
example, questions for the “French and Indian War” 
text segment queried: who fought the war, where the 
war took place, why there was a war, who won the war; 
and what was achieved by winning. 

Scoring. To establish a procedure for scoring recall 
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protocols, wc created a textbase based on a narrative 
analysis technique developed by Omanson (1982) and 
used in our previous work (Beck et al., 1984; Beck, 
Omanson, & McKeown, 1982). For a full discussion of 
the development of the textbase, see Beck et al. (1991). 
Omanson 's analysis identifies clauses that portray an 
event or state as content units. We adapted the analysis 
for use with expository text, which resulted in our 
identifying as units sequences that were often smaller 
than clauses. The procedures for developing the ton- 
base involved dividing the text into clausal units, estab- 
lishing separate units for any clause or phrase that 
represented a distinct state, event, location, or qualifier 
Because of the elliptical nature of the original ton, 
some implied units were added where inferences were 
deemed necessary for understanding. 

Procedures for scoring recalls . Each student’s re- 
call was scored for the gist of each text unit. Scoring 
was done by one of the researchers, and the scoring 
procedure has been shown to be reliable at 9 1 % agree- 
ment (Beck et al., 1991). 

Procedures for scoring questions. Questions were 
scored on the basis of model answers developed by the 
investigators. Of the 34 questions across the four seg- 
ments, 28 were scored either 0 or 1 . Half credit was 
possible for the remaining 6 questions. As with recall, 
scoring was done by one of the researchers and was 
shown to be reliable at 94% agreement (Beck et al., 
1991). 

Quantitative results 

The major question in the present study was how 
making relevant background knowledge available 
would affect the comprehension of more and less co- 
herent versions of a text. In this section we present the 
results of the statistical analyses of comprehension per- 
formance on the original and revised texts. 

Comprehension effects 

The data for recall of units common to both texts 
and the answers to questions were examined in separate 
mixed design analyses of variance (ANOVAs) using text 
type (original versus revised) as the between-subjects 
factor and passage segment ("French and Indian War;" 
‘‘No Taxation "Boston Tea Party," and "Intolerable 
Acts") as four levels of a within-subjects factor 
Insert Tible 1 about here 

TUblc 1 shows that students who read the revised 
text recalled a greater percentage of content units than 
students who read the original text, and this difference 
was significant, F{ 1, 46) - 7.59 , p < .01. This compre- 
hension advantage was demonstrated in students’ ques- 



Tabte 1 Mean percentages (and standard deviations) 
of units recalled and questions answered 
correctly 



Recall* Questions* 1 

Original Revised Original Revised 



French A Indian 
Wit 


133 

(7.7) 


19.3 

(11.0) 


40.6 

(25-3) 


42.7 

(25.0) 


No Tixation 


11.4 

(8.6) 


13.4 

(9.1) 


336 

(183) 


47.5 

(22.8) 


Boston Tfca Party 


26.2 

(12.8) 


33.2 

(16.3) 


50.8 

(24.9) 


630 

(29.0) 


Intolerable Acts 


14.7 

(12.8) 


315 

(20.3) 


29.2 

(27.7) 


60.9 

(36.0) 


Total 


16.4 


24.8 


38.5 


535 



R The number of units for each lew segment was J8. J8. 27; 21. respectively. 
t» The number of questions for each text segment was H. 9. 8. 9. respectively. 



tion-answering performance as well. Tible 1 shows that 
students who read the revised text answered more ques- 
tions correctly. This difference was also significant, F{ 1 , 
46) * 6.08, p < .05. These findings essentially repli- 
cate the findings of Beck et al. (1991) in that they show 
a comprehension advantage for the revised text materi- 
als. 

Passage effects 

There were significant differences in comprehen- 
sion performance among passages for both recall and 
questions, recall F(3, 138) - 34.70 , p < .01; questions 
^(3, 138) - 7.7 1 , p < .01. There was also a significant 
text condition by passage interaction for both recall, 
138) - 5 37 , p < .01, and question performance, 
P{ 3, 138) ■ 5.20 , p < .01. (Figures 1 and 2 graphically 
depict the data in Table 1 .) 

Insert Figures 1 and 2 about here 
As can be seen in Figure 1 , for both the original 
and the revised groups, the lowest recall performance 
was for the "No Taxation" text segment and the highest 
was for "Boston Tea Party." The significant interaction 
reveals, however; that the advantage for the revised 
group is not equal across the four text segments. The 
difference between the two groups' recall performance 
on the "Intolerable Acts" segment is more than twice 
that of the differences between their performance on t 
the other three segments. 

The results for the question performance show a 
somewhat different pattern. Both groups showed high- 
est question performance on "Boston Tea Party." How- 
ever, for the original group the lowest question 
performance was for the "Intolerable Acts" segment, 
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Figure 1 




IhdanWor T«a Party Act* 

Text passages 



while for the revised group, the lowest question per- 
formance was for “French and Indian War.” Further, as 
indicated by the interaction depicted in Figure 2, the 
difference in performance of the two groups on the 
“French and Indian War” passage is not comparable in 
magnitude to the comprehension advantage seen for 
the revised group on the other three passages. 

The pattern described may be a consequence of the 
interaction of the strong schema for Britain and the 
colonies established in the knowledge module and the 
revised text’s more extensive foreshadowing of the up- 
coming Revolutionary conflict. Because the most salient 
part of the discrepant patterns is the relatively poor 
performance of the revised group on the questions for 
the “French and Indian War” segment, we looked at 
the responses to the eight individual questions and 
found that the revised group had relatively poor per- 
formance on three of the questions. Specifically, the 



Figure 2 

Percentage of questions answered correctly for each 
passage 




Text passages 



questions concerned who had the land before the war, 
what Britain got from the French and Indian War, and 
how the colonies were getting along with Britain right 
after the war. In regard to who had the land before, the 
most common incorrect response was the Indians, and 
we cannot speculate as to why this occurred. 

As for the two other questions, however, the re- 
vised group’s responses indicated that they may have 
accessed their knowledge about the British and colonial 
antagonism gained from the knowledge module too 
early in the text sequence. That is, they often re- 
sponded that, the British had won the colonies as a 
result of the French and Indian War and that the British 
and colonists were not getting along well after that war. 

Responses to the three questions discussed reduce 
the advantage for the revised text. For the subsequent 
three text segments, however, the difference in favor of 
the revised text is much more apparent. 
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Qualitative results: A closer examination 
of knowledge effects 

The results of this study indicate a dear advantage 
for the revised text and therefore suggest that a rather 
extensive prior knowledge lesson that introduced infor- 
mation relevant for comprehending the text did not 
compensate for the lack of coherence in the text. How- 
ever, finding that prior knowledge provided no en- 
hancement for comprehension would be in contrast 
with arguably the strongest, most consistent finding in 
recent reading research (see, for example, Roller; 1990). 
Thus, the results warrant a closer look. 

An examination of the results from the present 
study in conjunction with the results of Beck et al. 
(1991), which originally found a comprehension advan- 
tage for the revised texts without a knowledge compo- 
nent, reveals differences that may shed some light on 
the, in this case, elusive effect of prior knowledge. Al- 
though many cautions need to be made when making 
comparisons across studies, we believe we have a strong 
case for why it is legitimate to compare the data of 
these two studies in particular. First, the samples are 
similar in that the students came from the same small, 
stable school district, they had the same social studies 
curriculum and the same fifth-grade social studies 
teacher, and their ability levels as shown by their read- 
ing comprehension achievement test scores were very 
comparable. In the present study, the mean comprehen- 
sion score was 66.44 (SD « 233) and in Beck et al. 
(1991)61.6(50 ■ 22.3). Further, the conditions of the 
studies were similar in that the same experimenter was 
in charge of running the subjects and scored the recalls 
and question responses for both studies. 

Most compelling is the fact that the pattern of dif- 
ferences in performance across studies is well ac- 
counted for by the presence or absence of the factors 
investigated in the studies, background knowledge and 
text coherence. This pattern, in conjunction with the 
similarity of the samples and conditions of the studies, 
seems to rule against reasonable alternative hypotheses 
for the observed differences. Additionally, no event that 
could significantly differentiate the history of the sub- 
jects in the two studies was noted, nor could we hy- 
pothesize one that would pose a competing explanation 
for the pattern of differences observed. 

The pattern of differences across the studies creates 
a revealing picture of the differential effects of knowl- 
edge and text coherence. In order to consider these 
patterns, we first present a brief review of the results of 
Beck et al. (1991), hereafter referred to as the revision 
study. 



Table 2 Mean percentages (and standard deviations) 
of units recalled and questions answered 
correctly by students in the revision study 
(Beck etai., 1991) 





Recall* 


Questions^- 




Original 


Revised 


Original 


Revised 


French & Indian 
War 


11.3 

(8.5) 


14.6 

(12.1) 


30.9 

(26.0) 


50.8 

(27.1) 


No luxation 


8.3 

(80) 


13 0 
(HO) 


22.4 

(24.2) 


38.3 

(26.5) 


Boston Tea Party 


21.6 

(12.2) 


26.6 

(12.0) 


41.9 

(31.0) 


61.7 

(23.1) 


Intolerable Acts 


13.1 

(11-5) 


17.9 

033) 


24.3 

(29.3) 


45.6 
(33 0) 


Total 


1.3.6 


18 0 


29.9 


49.1 



J The number of unit* for each text segment was 58. 38. 27. 2 1 . respectively. 
® The number of questions for each text segment was 8. 9. 8. 9. respectively. 



The revision study 

The priraaiy question in the revision study was 
whether the revised text produced better comprehen- 
sion than the original text. As Tiblc 2 indicates, the 
results of this study showed that students who read the 
revised text recalled a greater number of content units 
from the text and answered more questions correctly 
than the students who read the original text. These 
differences were significant. 

Discerning patterns of facilitation of knowledge 
and coherence 

We can begin our quest for the possible effects of 
prior knowledge on comprehension by comparing the 
data in Table 1 to the data in Table 2. As indicated, the 
percentage of units recalled and questions correctly 
answered is an average of 5% higher for the groups in 
the present study than for their counterparts in the 
revision study. Thus there is some suggestion of an 
overall facilitative effect of the prior knowledge compo- 
nent. But this overall view is not the comparison of 
primary interest. Rather; the most compelling evidence 
for the effects of knowledge and text lies in the pattern 
of differences across the four groups in the two studies, 
particularly in students' responses to specific questions. 

The pattern of differences suggests where knowl- 
edge from the module facilitated comprehension of the 
original text, where that facilitating effect was limited, 
and where the combination of information presented in 
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‘fa b le 3 Percentage of target questions correctly answered for the “No Taxation Without Representation text 
segment 



Text condition: 


Original text 




Revised text 


Knowledge module: 




♦ 




♦ 




OTa. OT/K 


RTk. 


RT/K 


Questions targeting agents and attitudes 


I. Who is being asked to pay taxes? 


55 


92 


71 


08 


2. How do the colonists feel about the taxes? 


48 


79 


71 


96 


Questions targeting representation principle 

3. What does it mean that the colonists were not members of Parliament? 


10 


8 


38 


46 


4. Why are the colonists upset about the taxes? 


8 


13 


16 


33 


5. What does “no taxation without representation “ mean? 


13 


17 


40 


54 



a, Dan in thne column* are teued on dan reported In Bcckcc al. (1991). 



the knowledge module and more coherent text pro- 
vided enhanced facilitation. These effects are best re- 
vealed If we examine two critical aspects of text 
content, which had been identified earlier in our work 
(McKeown & Beck, 1990) as lacking in students’ back- 
ground knowledge repertoires. These aspects involved, 
first, the role of the major agents— the colonies and 
Britain— and their attitudes toward key events, and, 
second, the principle that drove the colonists’ actions, 
representative government. As noted earlier, the knowl- 
edge module emphasized these two aspects. Informa- 
tion about the agents and their roles was directly 
presented to help students set up a conflict framework 
as they read; information about Britain’s governance of 
the colonies and the colonists’ powers of self-govern- 
ment was presented to set up a framework in which to 
understand the colonists' desire for representation. 

Included in the sets of questions that we developed 
for the texts were questions that targeted both of these 
aspects. In this section we organize the discussion 
around these target questions for three of the four sec- 
tions of text used in the revision study and the present 
study. Here we omit discussion of the “French and In- 
dian War’’ section because we have already discussed 
hypotheses for the effect of the knowledge module on 
the readers of the two different versions of that text 
segment. In particular, it appeared that the information 
about agents in the knowledge module, in conjunction 
with the more extensive foreshadowing of the Revolu- 
tion in the revised text, may have caused the readers of 
the revised text to access their Revolutionary schema 
too early in responding to questions. Additionally, the 



questions for the “French and Indian War” segment are 
the least relevant for the two aspects we are examining: 
the agents and their attitudes and the principle of repre- 
sentative government. The intent of the “French and 
Indian War” text segment was to set up a situation from 
which conflict between Britain and the colonies eventu- 
ally grows, that situation being Britain’s taxation policy. 
But this segment itself docs not set up that conflict, nor 
docs it concern the representation issue. Thus the ques- 
tions for the segment do not reveal the pattern of differ- 
ences that we focus on in this discussion. 

No taxation without representation. For the “No 
Tkxation” section we focus on five questions, two of 
which concern the identity of the major agents and 
their attitudes toward each other and three of which 
tap understanding of the principle motivating the colo- 
nists’ actions. These two sets of questions and the per- 
centage of correct response to each by the four groups 
are shown in “fable 3. 

We first consider the agent questions (Questions 1 
and 2), which query who is being asked to pay taxes 
and how the colonists feel about them. Performance on 
these two questions reveals the facilitating effect of the 
background knowledge module. That is, students who 
read the original text and received the knowledge mod- 
ule (the OT/K group) demonstrated a large advantage 
over those students who read the original text without 
the knowledge module (OT). Specifically, for the first 
question, the OT/K group demonstrated an advantage 
of 39% (92% versus 53%), and for Question 2, 31% 
(79% versus 48%). A similar advantage was shown on 
Questions 1 and 2 for those students who read the re- 
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Ihble 4 Percentage of target questions correctly answered for the "Boston Tea Party" text segment 



Text condition: Original text Revised text 

Knowledge module: — «. _ 



OT% OT/K RT* RT/K 



Questions targeting agents and attitudes 



1 . What was the Boston Tea Party? 


73 


92 


80 


80 


2. Who took part in the Boston lira Party? 


SO 


83 


78 


88 


Questions targeting principle 

3- Why were they upset about a small tax? 


25 


33 


44 


46 


4 Why did they throw tea into the water? 


28 


38 


49 


58 



& Data in these columns arc based on data reported in Beck ct al. (1991). 



vised text and received the knowledge module (RT/K) 
over those students who read the revised text without 
the knowledge module (RT). For Question 1 this differ- 
ence was 17% (88% versus 71%) and for Question 2, 
25% (96% versus 71%). This advantage suggests that 
students were using information from the knowledge 
module to help them distinguish the roles of the agents 
in the text and perhaps also drawing on information 
from the module about Britain’s governance of the col- 
onies, that is, that Britain could pass laws that the colo- 
nies had to follow. 

A somewhat different picture of the effect of 
knowledge emerges when we consider the questions 
involving the principle underlying the colonists’ actions 
(Questions 3, 4, and 5). The scores were very similar for 
the two groups who read the original text. Thus, it 
seems that students who had the knowledge module 
(OT/K) were not able to apply that knowledge to better 
understand the basis for the colonists’ distress over 
Britain's taxes. In fact, across the three questions that 
tap the representation, principle, the highest percentage 
of correct responses to any of these questions for the 
two groups that read the original text is only 17%. In 
contrast, knowledge combined with the greater coher- 
ence of the revised text did seem to play a role in how 
students responded to these three questions. The RT/K 
group shows advantages ranging from 8 to 17% over 
the RT group for the three questions that targeted the 
principle. 

It is possible to consider whether there were sepa- 
rate effects of knowledge and coherence by comparing 
the performance of students who read the revised text 
without the knowledge module (RT) to those who read 
the original text with the knowledge module (OT/K). 
Coherence seems the stronger influence, as shown by 



the performance of the RT group on the questions in- 
volving the principle motivating the colonists. 

Boston Tka Party. The questions for the "Boston 
Tea Party" segment that tap the agents’ roles and the 
motivating principle underlying the colonists’ actions 
are shown in Table 4. As in the "No Tixation" section, 
students who read the original text and received the 
knowledge module (OT/K) showed higher performance 
on the agents and attitudes questions (Questions 1 and 
2) than students in the OT group, but that advantage 
did not extend strongly to the questions involving the 
principle. Specifically, the advantage for Question 1 was 
19% for the OT/K group (92% versus 73%) in under- 
standing that the tea party was the colonists’ action of 
throwing tea in the water. For Question 2 the advantage 
was 33% (83% versus 50%) in understanding that it 
was the colonists who took pan in the tea party. Inter- 
estingly, these advantages are for two questions that 
were relatively well responded to by the OT group. 

Thus the advantage is shown for two of the easier, more 
basic questions. 

For students who read the revised text, the effect 
of the addition of knowledge is not dear for the agent 
questions. For Question 1, the scores of the RT and 
RT/K groups arc identical, while for Question 2, which 
is a question based on essentially the same content, the 
RT/K group showed a 10% advantage. It can be noted 
that for these two groups, as well as for the OT/K 
group, performance is quite high, with more than three 
quarters of the students correctly responding to both 
questions. 

For this text segment, Questions 3 and 4 relate to 
the principle. Question 3 asked why the colonists were 
upset about a tax that was described in the text as 
"small"— in order to see if students realized it was not 
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Thble 5 Percentage of target questions correctly answered for the 4 4 Intolerable Acts’ ’ 


text segment 




Tkxt condition: 


Original text 


Revised text 


Knowledge module: 


— 


+ 


— 


+ 




OTl 


OT/K 


RT J 


RT/K 


Questions targeting agents and attitudes 










1 . How did the British feel about the Boston Tea Party? 


56 


88 


88 


90 


2. Why did they close the Port of Boston? 


15 


25 


58 


50 


3. What was intolerable? 


15 


21 


40 


67 


4. How did the people in the other colonies feel about what was 


25 


21 


56 


63 


happening in Boston? 










Questions targeting principle 










5. What does it mean that no self-government was allowed? 


15 


25 


27 


63 



ftDaa in these columns a»e based un dan reported in Bechet al. (1991). 



the amount of money that bothered the colonists but 
that the tax was being levied without their say. Ques- 
tion 4 queried the colonists’ reasons for throwing tea 
into Boston Harbor to tap whether students understood 
it as an act of protest against Britain’s taxes. The OT/K 
group shows only modest advantages over the OT 
group on these questions. For Question 3, there was an 
6% advantage, and for Question 4 there was a 10% 
advantage. 

A similar picture of, at best, a modest advantage 
holds for the effect of knowledge in addition to the 
revised text for the two principle questions. Specifi- 
cally, for Question 3 the RT and RT/K groups show a 
nearly identical score, and for Question 4 the RT/K 
group shows a 9% advantage. Further; these two ques- 
tions indicate the stronger influence of text coherence 
over background knowledge. That is, both groups who 
read the revised text show advantages ranging from 1 1 
to 20% over the OT/K group. 

In general, the pattern of question responses for 
the "Boston Tea Party’’ segment suggests that it was 
fairly easy to help students understand that the colo- 
nists were the main actors in the events described in 
this text segment, but it was much more difficult to 
upgrade their understanding of the principle behind the 
colonists' actions. Yet the enhancements of knowledge 
and tot coherence did provide assistance, even though 
the magnitude of that assistance was not large. And, a 
pattern emerges that suggests that knowledge helped 
readers of the original text, text coherence provided 
more help than the addition of knowledge to the origi- 
nal text, and knowledge in addition to coherence pro- 
vided the most facilitation. 



Intolerable Acts. For the "Intolerable Acts" seg- 
ment, we again consider questions that focused on the 
agents’ roles and attitudes and on the principle of repre- 
sentation in government. The questions and the per- 
centage correct for each group are shown in Tible 5. 
But, as will be discussed shortly, the distinction be- 
tween the two categories of questions is less clear in 
this segment. Question 1 is the clearest example of a 
query about the attitude of a major agent, that is, how 
the British felt about the Boston Tea Party. In keeping 
with the pattern that is emerging, there is a striking 
32% advantage for the OT/K group (88% versus 56%) 
over the OT group. 

The RT/K group did not show such an advantage 
for Question 1, but this likely represents a ceiling effect, 
as the RT group’s response was already 88%. For Ques- 
tion 2, which asks about an action that stemmed from 
Britain’s attitude about the tea party, an advantage is 
also shown for the OT/K group, although it is much 
smaller (10%). The RT/K group shows a similar advan- 
tage over the RT group for Question 2. 

The remaining two agent and attitude questions 
(Questions 3 and 4) have a rather different quality in 
that they require more than the realization that Britain 
and the colonies are on opposing sides in a conflict. For 
example, the answer to Question 3, "What was intoler- 
able?" is that "intolerable" is what the colonists 
dubbed Britain’s new laws. Understanding why the 
colonists used this term involves some realization of the 
intensity of the reaction of the colonies to the laws. The 
colonists’ strong reaction sprang from their belief that 
they had just cause for what they had done, because 
Britain had violated a principle that they held dean 



BEST COPY AVAILABLE 



17 



23 



Volume 2, Spring 1997 



Mill Reading Comprehension 



READING RESEARCH QUARTERLY 1992 27/1 



Question 4, which asked, “How did people in other 
colonics feel about what was happening in Boston?" is 
also a complex question. It requires understanding that 
the colonies were not one single entity in opposition to 
Britain, but a collection of entities that were drawing 
together and developing a united opposition. 

For both Questions 3 and 4, the enhanced knowl- 
edge of the OT/K group was not sufficient to allow 
students to reach the more complex understandings 
needed to respond to these two questions. But knowl- 
edge in combination with the greater coherence of the 
revised text had a powerful effect, as the RT/K group 
outpaced the RT group by 27% on both questions. A 
comparison of the RT group to the OT/K group shows 
that text coherence had a greater effect than knowledge 
on the understandings needed for these complex ques- 
tions. That is, the RT group showed an advantage over 
the OT/K group on both questions. 

Question 5, which asks about the meaning of Brit- 
ain’s disallowance of self-government, is the only ques- 
tion directly involving the principle of representation 
for this text segment. Although the OT/K group shows a 
10% advantage over the OT group in response to Ques- 
tion 5, even with this advantage only a quarter of the 
students responded correctly. This brings the OT/K 
group’s response level in line with that of the RT group. 
The effect of knowledge in addition to coherence, how- 
ever, brings a striking advantage for the RT/K group, 
with 63% of the students giving a correct response. 

This is a powerful illustration of the effect of the combi- 
nation of background knowledge that the colonists had 
had some powers of self-government with a coherent 
explanation in the text that some of their rights were 
taken away. 

The overall pattern. It seems clear from consider- 
ing the pattern of differences among the groups in re- 
sponse to questions about key aspects of text content 
that enhancement due to background knowledge was 
not an overall general effect. Rather, knowledge af- 
fected the original and revised texts quite differently. 
When questions required distinguishing the basic roles 
of the major agents or discerning their attitudes toward 
key events, the knowledge module played a strong role 
for readers of the original text. In contrast, the knowl- 
edge module never provided a strong advantage for 
these readers in understanding the principle underlying 
the colonists’ actions or in comprehending more com- 
plex aspects of the agents’ roles. At most, modest ad- 
vantages were seen for some of these questions. 

For readers of the revised text, knowledge played 
at least a moderate role for questions concerning 
agents, except in two cases where correct response was 
already at 80%. Questions involving principle or more 



complex aspects of agents’ roles generally showed 
larger advantages for these readers than for readers of 
the original text. 

In comparing the knowledge effect to the coher- 
ence effect, as embodied in the OT/K and the RT condi- 
tions, coherence seems the stronger enhancement for 
understanding complex content. The two enhance- 
ments together, in the RT/K condition, seem particu- 
larly potent in this regard, often providing an advantage 
b^ond the sum of what the two provide separately. 
This large advantage is especially evident for a number 
of questions in the "Intolerable Acts’’ text, the Final 
segment in the sequence. It is difficult to trace the ad- 
vantage of the RT/K group for questions on this seg- 
ment to specific information from the knowledge 
module, or to determine how specific information 
might have interacted with certain text statements. 
Rather, this advantage may suggest an accumulating 
effect of the two enhancements, knowledge and coher- 
ence, that allowed the RT/K group to construct a more 
complete representation of the event sequence being 
portrayed. 

Discussion 

In investigating the relative contribution of knowl- 
edge to comprehension of more and less coherent text, 
we anticipated either of two possible outcomes. The 
first was that upgrading students’ background knowl- 
edge would override the comprehension advantage for 
the revised text. The rationale for this possibility is that, 
given appropriate background knowledge, students 
could bring that knowledge to bear on the rather ellipti- 
cal nature of the original text and make the connecting 
inferences needed to bring coherence to the material. 
This line of thinking coincides with Roller’s (1990) 
discussion of the interaction of text structure and back- 
ground knowledge. That is, clear and logical structure 
of a text— which can be likened to our concept of co- 
herence in many respects— is most useful to readers 
encountering text in moderately unfamiliar domains. 
Thus it might be posited that the knowledge module 
would have provided enough familiarity with the con- 
tent to override the need for the kind of structural sup- 
port present in the coherent text. 

The other possibility was that background knowl- 
edge in addition to more coherent text would result in 
better comprehension than coherent text alone. The 
notion here is that, given the conceptual difficulty of 
the material, two sources of support, coherent text and 
background knowledge, would give rise to better com- 
prehension rather than either source overriding the 
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effect of the other. 

As is clear from the results, it was this second possi- 
bility that prev ailed. The students who read the revised 
text were able to utilize the knowledge gained from the 
background knowledge instructional module to focus 
on and remember the most important information from 
the text. This was particularly evident from the exami- 
nation of responses to questions. The revised group was 
most successful in responding to key questions that 
indicated some grasp of the principles motivating the 
colonists’ actions. 

The students who read the original text, although 
they received the same background information, were 
less able to exploit the advantage provided by that in- 
formation. It seems that the nature of the original text 
prevented students from bringing their knowledge to 
bear in constructing meaning from the text. The effect 
of upgraded knowledge on the readers of the original 
text seems to be the other side of the coin to Roller’s 
(1990) argument that text structure has differential ef- 
fects for different levels of knowledge. That is, the 
present results suggest that background knowledge is 
most useful if the text is coherent enough to allow the 
reader to sec the connections between the text informa- 
tion and previous knowledge so that the knowledge can 
be combined with the text information to create a 
meaningful representation. 

Regarding the effect on text comprehension of the 
knowledge provided by the background knowledge 
instructional module, one additional finding begs fur- 
ther comment. The finding, which was presented ear- 
lier, is that of the comparatively poor performance of 
the revised group on the questions about the French 
and Indian War. As was mentioned previously, the hy- 
pothesis is that this finding is traceable to the emphasis 
created in the instruction on the diverging identities of 
the actors in the Revolutionary War, which likely drew 
attention away from the specifics of the French and 
Indian War. This finding is worthy of note because it 
represents an example — created inadvertently — of a 
phenomenon that is currently prominent in the litera- 
ture on background knowledge. The phenomenon is 
that of background knowledge overriding text informa- 
tion in such a way that readers recall text or respond to 
questions with information from their knowledge base, 
even though it is not supported by — and in some cases 
is even contradictory to— the text (Alvermann, Smith, & 
Readencc, 1985; diSessa, 1983; Dole & Smith, 1987; 
Upson, 1982; Marshall, 1989; Schoenfeld, 1985). In the 
case at hand, it appears that students depended on their 
either newly acquired or newly activated schema about 
the Revolutionary period to answer questions about the 
earlier war presented in the text. 



The research reported here also addresses the issue 
of the contribution of textbooks to social studies learn- 
ing, particularly in relation to the teacher s role in 
mediating text information. Elementary-text -based les- 
sons — be they in reading, science, or social studies — as 
represented in the teacher’s manuals that accompany 
textbook series are initiated with a preparation compo- 
nent. Preparation components are teacher-initiated 
activities, often teacher-led discussions, through which 
the teacher attempts to provide the students with skills 
and background knowledge related to the upcoming 
textual material. The information provided might in- 
clude, for example, a context for a story, the meanings 
of some vocabulary, or explanation of a concept. It is 
conventional wisdom that the teacher has a significant 
role in “preparing” elementary students for the textual 
materials they will encounter, although there are differ- 
ences in what is recommended across grade levels, text- 
book series, and subject matters, as well as individual 
differences among teachers. 

One such difference across subject matters is appar- 
ent in even a cursory comparison of teacher’s manuals 
for reading and social studies. The suggestions in the 
preparation component for social studies are far less 
extensive than those found in reading textbook series. 
The more extensive preparation components for read- 
ing selections may have developed because, given that 
reading is a process and not a subject matter, what stu- 
dents can read about in a reading program can be cho- 
sen from the multitude of topics in the world (Beck & 
McKeown, 1989). Thus, because reading programs tend 
to be a smorgasbord of content with frequent, often 
daily, changes in topic, there is obvious need to provide 
background. In contrast, the less extensive preparation 
component in the social studies teacher’s manuals may 
have its roots in the notion that a content area textbook 
is by definition sequentially developed so that early 
chapters serve as background for subsequent chapters. 

The results of this study in combination with those 
of the revision study (Beck et al., 1991) suggest that the 
notion of sequenced content as background, at least for 
social studies texts as they are now written, is a falla ry 
Previous text content as background, represented in the 
original text condition of the revision study, fell far 
short of preparing students for understanding text con- 
tent about events leading to the Revolutionary War. This 
result demonstrates the need for teacher intervention, 
and rather substantial intervention at that. As discussed 
in Beck et al. (1991), the original group received regular 
classroom instruction accompanying the content lead- 
ing up to the tested material, which included teacher 
intervention at least at the level prescribed in the 
teacher manual. 
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Another wrinkle in the prescription of teacher 
intervention is revealed by virtue of the performance of 
the two groups in the present study. That is, the original 
textbook passages, coupled with fairly extensive 
teacher intervention to provide background knowledge, 
yielded comprehension that was below that of students 
who received the preparation component and the re- 
vised text. This result counters another argument some- 
times put forth in the debate on textbooks; that is, 
efforts to make textbooks more coherent are unneces- 
sary because the teacher fills in much information 
which will make up for many gaps in the text material. 
As the research reported here has shown, extensive 
preparation to provide background knowledge did not 
compensate for the inadequacies of the text; there was 
still a substantial advantage for the revised text. 

This study has shed some additional light on the 
contribution to text comprehension of two important 
components of that process, background knowledge 
and text coherence. As Roller (1990) has pointed out, 
text structure and background knowledge interact in 
their effect on comprehension. Knowledge of varying 
degrees can compensate for varying levels of struc- 
ture— or, in the present study, coherence. Similarly, 
greater coherence can compensate for some knowledge 
gaps. Our portrayal of interaction is one in which both 
variables are individually effective, but neither variable 
can completely compensate for inadequacies in the 
other. 
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ABSTRACT 



The purpose of this study was to describe the remedial reading instruction received 
by students assigned to inclass or pullout programs with respect to the instructional 
behaviors of teachers, the nature of lessons, and the reading behaviors of students. 

The study was conducted in an urban school system in which the pullout and 
inclass programs existed concurrently. One hundred nineteen students (Grades 4 
and 5) were observed over a 4-month period. In both settings, students received 
a great deal of skill-related instruction; not much time was spent in prereading 
activities and students spent most of their time with worksheets. Over 60% of 
classroom time in both settings was spent in listening, transcribing, and silent 
reading. Pullout students experienced more direct teaching activities in a group 
situation while inclass students experienced more time in a one-on-one tutoring 
mode. Overall, there were differences between settings in what students 
experienced in remedial reading. 

Many concerns have been raised about the prevailing model used in Chapter 
I programs— the pullout model that provides supplemental compensatory instruc- 
tion to participating students. Critics of pullout programs question the lack of 
congruence with the developmental reading program (Allington & McGill-Franzen, 
1988; Allington, Steutzel, Shake, & Lamarche, 1986; Johnston, Allington, Sc Af- 
flerbach, 1985), the loss in instructional time in moving from classroom to pullout 
setting (Allington et al., 1985); and the negative effects of labeling students (Glass 
& Smith, 1977; Leinhardt & Pallay, 1982). 

Moreover, results of the latest national assessment of Chapter I have indicated 
that: (a) although students receiving Chapter I services demonstrated larger stan- 
dardized achievement test gains than comparable students not receiving Chapter I 
services, their gains did not move them substantially toward the achievement level 
of more advantaged students; (b) participants in math gained more than those in 
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reading; and (c) students in the early grades gained more than students in later 
grades (Kennedy, Birman, & Demaline, 1986). These results are consistent with 
the accumulating evidence and concern about the overall effectiveness of Chapter 
I (Carter, 1984; Cooley & Leinhardt, 1980; Coulson et al., 1977; Kaestle & Smith, 
1982; National Institute of Education (NIE), 1978; Slavin, 1987; Wang, 1980). 

Given concerns about pullout programs, several alternative models have been 
proposed, including an inclass model in which the reading specialist works with 
students in the classroom alongside the regular or developmental reading, teacher. 
Although proponents assume that the inclass model will eliminate problems such 
as lack of congruence, loss of instructional time, and labeling,' there is little evi- 
dence to support these assumptions. Indeed, there are concerns that the inclass 
model may generate new problems that may also reduce program effectiveness 
(Bean & Eichelberger, 1985; Harpring, 1985). 

The purpose of a larger study (Bean, Cooley, Eichelberger, Lazar, & Zigmond, 
1988), of which the work reported here is a part, was to investigate systematically 
the variables that appear to be critical to effective reading instruction as they occur 
in two different Chapter 1 programs, inclass and pullout, and to determine the 
effects of these variables on student achievement. In the larger study we looked at 
both developmental and remedial programs and the congruence between the two 
programs. In this paper, we focus our attention only on the remedial reading 
program with respect to the following questions: What is the nature of the remedial 
reading program for students in an inclass and a pullout setting? How do these 
programs differ in: (a) the instructional behaviors of the reading specialists, (b) the 
nature of the lessons, and (c) the reading behavior of students? We were especially 
interested in describing the nature of the reading instruction received by students, 
that is, how they were spending their instructional time when they were with the 
remedial teacher and when they were working independently. 



METHOD 



Subjects 

This study was conducted in a large city school system in which the two 
different settings for remediation existed concurrently. Schools within the district 
with smaller numbers of students and, therefore, fewer students eligible for Chapter 
I, used a pullout model for their Chapter I reading program, whereas the Chapter 
I reading program in larger schools was inclass in nature. The decision regarding 
setting was made at the district level and was related to difficulties in scheduling 
remedial reading teachers assigned to handle students in two or three schools. The 
objectives and philosophy as well as instructional techniques for the Chapter I 
reading program were consistent across the district, however, helping students to 
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be successful in the classroom reading program was emphasized as an important 
aspect of the Chapter I program, regardless of setting. The fact that we could study 
a Chapter I reading program that consisted of both inclass and pullout models in a 
single district enabled us to control for reading philosophy, materials, community 
characteristics, and teacher quality to a greater extent than if we were studying 
each type of setting in a different school district. 

One hundred nineteen students (Grades 4 and 5) participating in Chapter 1 
remedial reading programs in 12 schools (4 inclass; 8 pullout) were observed 
systematically over a 4-month period while receiving reading instruction from their 
remedial reading specialists. All students had scored at the 30th percentile or below 
on the California Achievement Test (1987), Total Reading, the previous spring. 
Of the 119 students, 66 students (27F, 39M) received remedial instruction in an 
inclass setting and 53 (20F, 33M) in a pullout setting. 

Nine reading specialists served these 119 students in 12 schools (one at 3 of 
the schools, another at 2 schools, and the remaining at one school each). The 
participating schools were scattered geographically throughout the school district. 
The level of poverty in participating schools ranged from 25% to 75% of students 
eligible for free lunch. The percent of school populations participating in Chapter 
I programs ranged from 5.0% to 9.4%. School sizes ranged from a total school 
population of 168 students to a school population of 476. 

In the pullout setting, students left their regular or developmental classroom 
at some time other than during reading instruction to receive instruction from the 
reading specialist. Specialists saw no more than 10 students at a time, and students 
received instruction for 100 minutes per week (two and one half 40-minute periods). 
In the inclass setting, the reading specialist went to the classroom during reading 
instruction two or three times per week, observed for half of the 40-minute period 
to determine the focus of the classroom teacher and possible needs of the students, 
and then worked with the identified students (no more than 10) for the remaining 
time (a total of 50 minutes of instruction per week). 

Materials 

Observational data on teacher behaviors, lessons, and student behaviors were 
obtained through the SORIN (System for Observing Reading Instruction), a time- 
sampling observation protocol that enabled the observer to specify the amount of 
time spent on designated instructional activities as well as on selected teacher and 
student behaviors. The coding system was designed by the researchers (Bean, 
Lazar, & Zigmond, 1987) to yield information on the students’ activities and 
experiences during the time allocated to reading instruction. Qualitative information 
about the specific materials used, nature and size of group, and content of lessons 
were also recorded. 

In developing the observation instrument, findings of prior research about 
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instructional practices that are related to reading achievement were used to provide 
initial observation categories (see Kohnke, Miller, & Zigmond, 1986; Leinhardt & 
Seewald, 1981). Further, three basic beliefs as to what constitutes effective reading 
instruction guided us in designing the instrument: (a) reading is more than a hierar- 
chically arranged set of subskills which can be systematically taught and then 
tested, (b) students need opportunities to experience reading of connected text, and 
(c) students who are experiencing reading difficulties profit from direct instruction 
designed to help them make sense of what they are reading (Adams, Anderson, & 
Durkin, 1984; Allington, 1984; Anderson & Scott, 1978; Bourque, 1980; Duffy, 
Roehler, & Mason, 1984; Paris, Lipson, & Wixon, 1983; Rosenshine, 1980, 1986; 
Solomon & Kendall, 1979; Winograd & Greenlee, 1986.) Our observation system, 
therefore, was designed to enable us to get a picture of students’ experiences 
relative to each of these beliefs. Table 1 illustrates categories for the three dimen- 
sions investigated, which are also described below. 

Although reading instruction is generally presented to groups of students, the 
activities and experiences of individual students often differ. For example, although 
one student may be reading orally, another may be listening; one student may be 
involved in a teacher/student dialogue while another is inattentive. Our goal was 
to describe the reading experiences of students in each setting; for that reason the 
unit of observation and analysis throughout the study was the individual student. 

Teacher behaviors . In the first phase of each observation cycle, students were 
observed to determine the nature of the teacher behaviors each was experiencing. 
The two basic areas of teacher behaviors recorded were instructional and nonin- 
structional. Given the focus on compensatory reading, instructional behaviors in- 
cluded such behaviors as: teacher working individually with one, student (individual 
contacts) and monitoring students while they completed an assignment (listening/ 
looking). Noninstructional behaviors were those behaviors that did not in any way 
pertain to instructional content or method (see Table 1). The observers systemati- 
cally observed each child for 5 seconds, then coded the teacher behavior experi- 
enced by that child during the next 5 seconds; then the observer followed the same 
procedure with the next child listed on the observation protocol. Student experi- 
ences of teacher behavior were coded 7 times, evenly spaced throughout the obser- 
vation period. 

Lesson. In the second phase of each observation cycle, data were collected on 
three dimensions of the lesson: lesson focus, types of materials used, and level of 
materials. Within the first dimension, no focus reflected that, for the student being 
observed, there was no discernible instructional focus at the time of the observation 
(e,g., student sitting around waiting for a lesson to begin while the teacher was 
organizing materials or writing on the chalkboard). A student was coded as being 
involved in a text-related lesson (TRL) if the instructional focus was centered on 
a selection of written text. These TRLS were further coded into activities occurring 
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Table 1 



Categories in the SORIN Observation System 




instructional No Focus Reading Related 



•Giving Information 
•Giving Directions 
•Questioning/ 
Answering 

-Reacting to Students 
-Inctividual Contacts 
-Listen! ng/Looklng 
-Inctirect 
Interaction 
-Feedback/ 
Acknowledgement 



Non-lnstuctional 

-Transition 
•No Interaction 
•Behavioral 



Text-Ralatad 

•Before 

-During 

-After 



SkilhRelatad 

-Word identiflcation 

•Vocabulary 

-Comprehension 

•Study/Testtaking 

-Composing 

-Speaklng/U stenlng 

-Spelfing/Grammar 



-Silent Reacting 

•Oral Reading 

-Composing 

-Transcribing 

-Listening 

-Talking 



Nonreadina 

-Volunteering 

-Waiting 

-Off-Task 



Management 

•NorvinstroctionaJ 

Interaction 



Independent Activities 

•Skill Activities 



-Free Reacting 
-Free Writing 



before, during, or after reading of text. Vocabulary development activities or devel- 
opment of prior knowledge before reading a text selection were considered before 
reading activities. Guided oral or silent reading or the asking of questions inter- 
spersed with reading were considered during reading activities. After reading activi- 
ties consisted of discussions or comprehension questions raised after a selection 
was completed. 

A student was coded as participating in a skill-related lesson (SRL) if the 
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teacher was providing direct instruction on various types of skill activities. Students 
working independently (IW), whether on skill assignments, free reading, or writ- 
ing, was recorded as a separate dimension. 

Students’ experiences with various types of materials were also coded. Observ- 
ers identified whether the student being observed was working with basal text, 
content text, tradebook, worksheet or workbook, or other material commonly used 
during reading instruction. 

The final dimension coded in the Lesson section of the SORIN was level of 
activity as reflected by the nature of the materials with which each student was 
working. The word level indicated experiences with materials which focused on 
letters or words in isolation; sentence or paragraph level materials focused on 
working with sentences or paragraphs (e.g., worksheets in which students were 
identifying the main idea or drawing conclusions after reading a short paragraph 
of text); selection level indicated experiences with narrative or expository text 
(including the followup discussion or completion of activities in which students 
were responding to questions about a selection). For the lesson dimensions, a 
student was observed for 5 seconds and then the appropriate codes for the focus, 
type, and level of materials were recorded. The observer then followed the same 
procedure for the next child on the observation protocol. Student experiences during 
a lesson were coded 7 times, evenly spaced during the observation period. 

Student behavior. The third phase of each observation cycle focused on what 
each student was doing during reading instruction. Included in this component were 
both direct reading behaviors (silent and oral reading) and activities related to 
reading (composing, transcribing, listening, or talking). In addition to these reading 
categories, three nonreading behaviors were coded (see Table 1). For this dimen- 
sion, a student was observed for 5 seconds and then during the next 5 seconds the 
observer coded the behavior. This procedure was then followed for the next child 
on the observation protocol. Student behaviors were coded 6 times, evenly spaced 
during a 40-minute observation period. 

Procedure 

Observations were conducted by 1 1 observers, all of whom were either reading 
specialists or had taught reading in some context and had a working knowledge of 
reading terminology. Observers were trained using an instructional manual, simula- 
tion experiences in a lab situation, and field experiences, until they reached criterion 
performance of 85% agreement with the trainer. After criterion was reached, inter- 
observer reliability checks were conducted every two weeks (15% of the total 
observations of each observer). Interobserver reliability remained high throughout 
the study (>.85). 

The observations of these 1 19 students took place over a 16-week period. For 
pullout programs, we observed during the complete 40-minute lesson period. For 
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inclass programs we observed the remedial instruction during the 20-minute seg- 
ments of two separate periods. (Additional observations of students* developmental 
classes were also made and are reported in Bean et al. v 1989.) For each remedial 
class, we sampled 40 minutes of instruction, every other week, for a total of 8 
periods of instruction (5 hours and 20 minutes) over the 4-month period. The 
decision regarding when to observe was based upon: (a) the schedule of the reme- 
dial teacher, and (b) the need to observe each child with both the remedial* and 
developmental teacher on the same day or on consecutive days to obtain measures 
of congruence (reported in Bean et al., 1989). 

Scoring . Based upon the raw counts obtained through SORIN, the frequency 
with which students experienced various activities was obtained and the mean 
percent for each frequency calculated. The time allocated for remediation in each 
of the settings (50 minutes a week for inclass students and 100 minutes a week for 
pullout students) was used to transform the mean percent of frequencies to the 
percent of time per week that students spent in specific activities and to determine 
how total amount of time in remedial reading was spent. 



Tables 2-6 list the percentages of remedial time students spent in various 
reading or instructional activities across the two settings. Differences in the percent 
of weekly time spent in each category were compared ( t tests are reported in Tables 
2-6). Given our interest in investigating the differences in time students spent in 
the various reading activities across setting and the need to know whether those 
differences were significant, statistical analyses were conducted. The relative sizes 
of the t statistics provide a basis for such comparisons. The significance level was 
set conservatively at .001 to minimize Type 1 errors that might result from the 
large number of comparisons made. In the following sections, we discuss the 
differences in the remedial instruction across settings for the three areas in 
the SORIN (teacher behaviors, lesson, and student behaviors). 

Teacher Behaviors 

The first section of the protocol permitted an analysis of the teaching behaviors 
experienced by students (see Table 2). In the pullout setting, students spent 78.1% 
of remedial reading time receiving instruction from the teacher and 5.7% of time 
in noninstructional activities. The remaining 16.1% of the time was coded as 
“scheduled instruction ceased,** that is, the lesson had been completed and students 
were involved in activities other than the specific reading lesson being observed 
(waiting for the next lesson to begin or waiting to return to their classrooms). In 
looking specifically at the instructional practices, students in the pullout remedial 
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Table 2 



Percentage of Remedial Reading Time Students Were Recipients of Various 
Reading Specialist Behaviors in Inclass and Pullout Settings 





Pullout 
Mean % 


SD 


Inclass 
Mean % 


SD 


t Value* 


Instructional 


Giving Information 


11.7 


5.4 


8.5 


9.2 


2.35 


Giving Directions 


7.3 


5.0 


2.9 


5.5 


4.43*** 


Questioning/ Answering 


10.4 


10.9 


5.3 


8.4 


2.85** 


Reading to Students 


4.6 


6.8 


1.9 


4.0 


2.51 


Individual Contacts 


5.1 


6.8 


10.8 


13.6 


-2.94** 


Listening/Looking 


21.2 


9.6 


10.2 


11.0 


5.77*** 


Indirect Interaction 


14.9 


11.5 


17.0 


14.2 


-0.89 


Feedback/ Acknowledgement 


2.9 


4.5 


0.5 


1.7 


3.72*** 


Total Instructional 


78.1 


9.2 


57.1 


13.2 


10.04*** 


Non instructional 


Transition 


1.6 


2.7 


4.2 


5.0 


-3.51*** 


No Interaction 


2.9 


5.5 


11.3 


17.1 


-3.69*** 


Behavior Management 


0.9 


1.6 


0.6 


2.2 


0.90 


Noninstructional Interaction 


0.3 


1.6 


1.1 


2.4 


-2.14 


Total Noninstructional 


5.7 


7.6 


17.2 


15.3 


-5.22*** 


Scheduled Instruction Ceased 


16.1 


6.5 


25.6 


13.7 


-4.93*** 


Out of Room 


0.0 


0.0 


0.4 


0.3 


-1.00 



•df=m. **p<oi. ***p< ooi. 



setting spent 21.2% of remedial reading time involved in activities in which their 
teachers were listening to them or watching them work: for example, students had 
been given a workbook assignment and the teacher was observing a specific student 
at work; student was responding to teacher questioning. Students were also involved 
in indirect interaction activities 14.9% of the time (the target student being observed 
was part of a group being taught, but the teacher was focusing on the group rather 
than on the specific student). Students also received information 11.7% of the time 
(e.g., “This is how you divide words into syllables”) and were involved in a 
questioning/answering dialogue with the teacher 10.4% of the time. Over 20% of 
the total student time in these pullout remedial settings was accounted for by: (a) 
no interaction (2.9%), (b) transition time. (1.6%), and (c) scheduled instruction 
ceased (16.1%). 

In the inclass remedial settings, students spent 57.1% of the time receiving 
instructional teacher attention, 17.2% in getting noninstructional teacher attention, 
and 25.6% when scheduled instruction had ceased. Most of the instructional time 
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was focused on indirect interaction activities (17.0%), that is, the target student 
was part of a group with whom the teacher was working but at the time of observa- 
tion, the teacher was not directing instruction specifically to the child being ob- 
served. Students also spent approximately 10% of their time in individual contacts 
with the teacher (teacher helping child with worksheet) or in listening/looking 
activities (teacher watching the child work or listening to the child respond). The 
greatest percentage of noninstructional time for students was in having no interac- 
tion with the teacher (1 1.3%), that is, the teacher was directly attending elsewhere. 
The combination of no interaction, transition (4.2%) and time coded as scheduled 
instruction ceased (25.6%) accounted for 41.1% of the total remedial time spent 
in the inclass setting. 

There was a significant difference between settings in the percent of time 
students experienced teacher instructional behaviors, with more time spent in in- 
struction in the pullout setting [t(\ 17)= 10.04, pC.OOl]. More specifically, sig- 
nificantly more student time in the pullout settings was spent on receiving direc- 
tions, listening/ looking interactions, and feedback acknowledgement (although 
little feedback was experienced by students in either setting). 

There were also significant differences between the two settings in the total 
percent of time coded as noninstructional [f( 1 17) = 5.22, pC.001]. There was sig- 
nificantly more time spent in remedial inclass settings in transition and in no inter- 
action , as well as in the total noninstructional time and in time coded as scheduled 
instruction ceased. The data do indicate greater variability in the inclass settings 
than in the pullout setting’s across several dimensions (e.g., no interaction, individ- 
ual contacts). 

Lesson 

In this area, three different aspects of the lesson were addressed: the lesson 
focus, the type of material used, and the level of that material (word, sentence, or 
selection) (see Table 3). 

Lesson focus. In the remedial pullout setting, students spent 50% of their time 
in skill-related lessons, 20.2% of their time engaged with text, and 7.1% of their 
time working independently. Skills emphasized were comprehension (15.8%), 
word identification (10.3%), and study skills (9.8%). Most text-related activities 
occurred during reading activities (15.2%). 

In the remedial inclass setting, students spent most of their time (39.4%) in 
skill-related activities, 20.8% of the time in text-related activities, and 8.4% in 
independent activities. During skill-related activities, students spent the greatest 
percentage of time in study skills/testtaking activities (11.7%) and in word identi- 
fication activities (10.4%). Students’ time when working with text was spent in 
after reading activities (9.7%) or in during reading activities (7.0%). 

There were no significant differences between pullout and inclass remedial 
settings in the percent of time that students spent on text-related work, although 
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Table 3 

Percentage of Remedial Time Students Spent on Aspects of Lesson Focus in 
Remedial Inclass and Pullout Settings 



Pullout Inclass 



Lesson Focus 


Mean % 


SD 


Mean % 


SD 


/ Value* 


No Focus 


1.4 


3.3 


2.3 


4.2 


-1.35 


Text Related 


Before Reading 


1.7 


2.8 


1.9 


4.9 


-0.28 


During Reading 


15.2 


14.2 


7.0 


12.1 


3.39*** 


After Reading 


2.9 


4.8 


9.7 


12.0 


-4.14*** 


Rereading 


0.1 


0.4 


1.6 


4.9 


-2.56 


Other 


0.3 


1.2 


0.6 


2.0 


0.97 


Total Text Related 


20.2 


16.8 


20.8 


22.2 


-0.16 


Skill Related 
Word Identification 


10.3 


11.4 


10.4 


12.9 


-0.07 


Vocabulary 


5.8 


7.7 


5.7 


8.5 


0.04 


Comprehension 


15.8 


14.5 


7.8 


8.3 


3.58*** 


Study Skills/Testtaking 


9.8 


14.6 


11.7 


18.1 


-0.60 


Spelling/Grammar 


1.7 


3.4 


1.9 


5.4 


-0.21 


Composing 


5.7 


10.4 


1.8 


5.1 


2.48 


Speaking/Listening 


0.9 


1.7 


0.0 


0.0 


3.68*** 


Total Skill Related 


50.0 


14.4 


39.4 


19.9 


3.24** 


Independent Work 


7.1 


9.9 


8.4 


10.1 


-0.71 


Scheduled Instruction Ceased 


20.0 


5.9 


28.7 


13.9 


-4.53*** 


Out of Room 


1.4 


6.0 


0.5 


2.3 


1.03 



*4T=117. *»p<.01. ***p<.00l . 



there were significant differences in how time was distributed within the various 
dimensions of that category. Students in the pullout remedial setting spent signifi- 
cantly more time in during reading activities [/(l 17) = 3.39, pC.001], whereas 
students in the inclass setting spent significantly more time in after reading activities 
Ml 17).= —4.14, p<.001]. 

Significantly more time was spent in remedial pullout settings on comprehen- 
sion skills [l(l 17) = 3.58, pC.OOl] and speaking/listening skills (although less than 
1% of the weekly time was spent on this skill in the pullout setting). The data 
again indicate greater variability in student experiences in the remedial inclass 
setting. 

Types of materials. In the remedial pullout setting, students spent most of their 
time with workbooks/ worksheets (34.3%) and tradebooks (13.0%) (see Table 4). 
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Table 4 



Percentage of Time Students Spent with Various Types of Materials in Remedial 
Inclass and Pullout Settings 



Material 


Pullout 
Mean % 


SD 


Inclass 
Mean % 


SD 


t Value* 


Basal 


5.5 


10.9 


20.5 


20.8 


— 5.01*** 


Tradebook 


13.0 


14.8 


0.3 


2.4 


6.16*** 


Content Text 


1.4 


4.2 


0.6 


4.2 


0.96 


Student Writing 


6.8 


8.7 


1.5 


4:2 


4.01*** 


Chalkboard/ Charts 


5.0 


10.3 


1.3 


3.7 


2.53 


Visuals/Tapes 


0.2 


1.1 


0.0 


0.0 


1.42 


Workbook/Worksheets 


34.3 


17.2 


35.6 


22.3 


-0.37 


Games/ Flashcards 


5.6 


8.9 


1.8 


4.5 


2.78*** 


No Materials 


2.1 


4.3 


2.2 


4.2 


-0.13 


Other 


5.0 


8.2 


7.0 


10.7 


-1.08 


Scheduled Instruction Ceased 


20.0 


5.9 


28.7 


13.8 


-4.57*** 


Out of Room 


1.3 


6.0 


0.5 


2.4 


0.90 



1 17. **p<.01. ***/><. 001. 



In the inclass remedial setting, students also spent most of their time with work- 
books or worksheets (35.6%) but also used basal materials (20.5%). 

The key differences were that students spent a significantly greater percentage 
of time in the remedial pullout setting with tradebooks [t(\ 17) = 6. 16, p<.001], 
student compositions [f(117) = 4.01, pC.OOl], and games and flashcards 
[r(117) = 2.78, pc.001], whereas students in the remedial inclass setting spent 
a significantly greater percentage of time with the basal reader [t(\ 17) = 5.01, 
pc.001]. 

Levels of text . In the remedial pullout setting, students spent more time with 
materials at the selection level (31.6%) than with materials focusing at the word 
level (23.0%) (see Table 5). Students in the remedial inclass setting spent more 
time with materials at the word level (28.9%) than with materials at the selection 
level (22.6%). There were no significant differences between settings on the percent 
of time students spent with different levels of text. 

Student Behaviors 

The three reading-related behaviors in which students spent most of their time 
in remedial pullout settings were listening (27.6%), silent reading (17.9%), or 
transcribing (16.7%) (see Table 6). The three reading-related behaviors that ac- 
counted for most of the students* time in the inclass remedial setting were also 
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Table 5 



Percentage of Time Students Spent with Various Levels of Text in Remedial 
Inclass and Pullout Settings 



Level of Text 



Pullout Inclass 

Mean % SD Mean % SD t Value* 



Word Level 


23.0 


Sentence/Paragraph 


19.3 


Selection 


31.6 


Not Applicable 


4.5 


Scheduled Instruction Ceased 


20.0 


Out of Room 


1.5 



20.5 


28.9 


17.8 


-1.67 


16.0 


16.6 


16.2 


0.92 


19.9 


22.6 


20.1 


2.44 


7.4 


2.7 


4.8 


1.53 


5.9 


28.7 


13.9 


-4.54*** 


6.1 


0.5 


2.3 


1.11 



*#=117. **p<.01. ***p<.001. 



listening (38.5%), transcribing (12.9%), and silent reading (8.6%). Figure 1 illus- 
trates graphically that these three activities accounted for over 60% of the students’ 
time in both settings. In the pullout setting the time was somewhat evenly distrib- 
uted among the three. 

Little time was spent in either setting on oral reading activities, nor was much 
time spent in composing, although more composing was found in the pullout set- 
ting. Students in the remedial pullout setting spent a greater percentage of time in 
silent reading than did students in the remedial inclass setting (17.9% vs. 8.6%), 
whereas students in the inclass setting spent a greater percent of time listening than 
did students in the pullout setting (38.5% vs. 27.6%). There was also a greater 
M^fe) 1 t * mC S * >Cnt t * 1C ' nC * 3SS sett * n 8 on volunteering to respond (3.5% vs. 



DISCUSSION 

Although prior research has indicated that setting itself is not a factor in pro- 
gram effectiveness or student achievement (Archambault, 1986; Leinhaidt & Pal- 
lay, 1982), setting did seem to have an effect on the nature of the instruction 
experienced by students in this study. Although there were clearly some similarities 
in the nature of the remedial instruction experienced by students in the two settings, 
there were also some striking differences. 

Similarities in Inclass and Pullout Settings 

One of the most frequently experienced activities in both settings was for 
students to receive instruction from the teacher as part of a group (indirect interac- 
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Table 6 

Percentage of Time Students Exhibited Various Reading-Related Behaviors in 
Remedial Inclass and Pullout Settings 



Reading-Related Behaviors 



Pullout 
Mean % 



SD 



Inclass 
Mean % 



SD 



t Value* 



Silent Reading 

Oral Reading 

Composing 

Transcribing 

Listening 

Talking 

Off Task 

Volunteering 

Waiting 

Scheduled Instruction Ceased 
Out of Room 



17.9 


17.9 


8.6 


17.0 


6.5 


6.2 


3.9 


7.6 


4.0 


9.0 


0.6 


2.4 


16.7 


11.1 


12.9 


13.4 


27.6 


18.3 


38.5 


21.4 


6.8 


6.0 


5.4 


8.6 


2.8 


4.4 


2.8 


5.0 


3.5 


4.7 


1.4 


4.1 


3.6 


4.8 


5.5 


8.5 


8.8 


8.2 


19.7 


12.7 


1.8 


6.5 


0.6 


2.4 



2 . 86 ** 

2.00 

2 . 66 ** 

1.66 

— 2.92** 
1.04 
- 0.01 
2.64** 
-1.58 
-5.61*** 
L25 



*df cs 117. **p<.01. +++p<Ml. 



tion) Moreover, in both, settings there was a large percent of time in which students 
were not involved in any instructional activities. In neither setting were students 
involved in many behaviors that required management activities on the part of 
teachers. Neither group of students received much feedback or acknowledgement, 
nor were students read to frequently. 

In terms of lesson focus, students in both settings expenenced relatively more 
time on skill-related tasks than on text-related activities. In both settings, a small 
amount of time was spent in independent activities, generally with students com- 
pleting workbook or worksheet assignments. This finding is different from that ot 
others (Anderson, Cook, Pellicer, & Spradling, 1989; Office of Educational Re- 
search and Improvement, 1987) who indicate that often students in remedial settings 
spend a great deal of time working alone on independent assignments. 

Instruction in both settings was distributed across materials at levels of the 
word, sentence/paragraph, and selection. In neither setting was much time spent 
in prereading activities. In both settings, a similar percent of time was spent in 
word identification activities, vocabulary, and study skills or test-taking activities. 

Students in both settings spent the greatest percent of their time listening, and, 
in both settings students experienced similar amounts of time in transcribing or 
copying activities. In neither setting did students spend much time in composing 
or in oral reading activities, nor was much time spent in off-task behavior, waiting, 
or volunteering. 
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Figure 1 . Percent of Weekly Time Students Exhibited Various Reading-Related 
Behaviors in Remedial Inclass and Pullout Settings 



Distinctive Features of the Pullout Setting 

The data from the pullout settings in this study in many ways confirm descrip- 
tions of others who have studied compensatory programs (Allington, 1986; Ailing- 
ton & McGill-Franzen, 1988; Archambault, 1986). A focus on skill instruction 
was predominant, although the impact of the recent research in reading which calls 
for more text-related activities could be seen in the use of tradebooks and student 
writing. There was relatively little time spent in activities prior to reading or after 
reading as compared to time during reading, however. This distribution of time does 
not reflect the current research which indicates a need for students to participate in 
prediction (Hansen, 1981; Olshavsky, 1976-77) or elaboration activities (Hidi & 
Anderson, 1986; Linden & Wittrock, 1981; Morrow, 1985). Moreover, the large 
percentage of time spent on listening as compared to reading, composing, or even 
discussion (talking) activities raises some questions about the appropriateness of 
these emphases in remedial classes for intermediate aged students. 

In general, the materials and selection of skills seemed unrelated to the reading 
instruction received in the regular classroom; reading specialists in the pullout 
programs tended to use supplemental materials and not the basal used in the devel- 
opmental reading program. In all pullout classes, instruction was implemented with 
the group as a whole. Thus, although these students may have been placed in a 
smaller group (size ranged from 2-5) than their classroom reading group, there 
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were few occasions where specialists individualized instruction for students within 
the group ! Moreover, often the reading specialists taught the same lesson to all 
groups of students they saw on a specific day or within a particular week. For 
example, during one week observers saw the same comprehension lesson, in which 
the goal was to help students become more familiar with story mapping as a means 
of enhancing comprehension, taught by one teacher with five different groups of 
students in five different classes. The same selection and worksheets were used 
with all classes. Although the goal of the lesson was a reasonable one for intermedi- 
ate aged students, there is some question about the appropriateness. of using the 
same selection with all of the students, across groups as well as within groups. 



Distinctive Features of the Inclass Setting 

In the inclass setting, individual students could frequently be found working 
with the reading specialist who functioned in a monitoring or tutoring mode. There 
were also greater amounts of time in which scheduled instruction had ceased or 
when there was no interaction between the reading specialist and students. A little 
more than a third of the instructional time was spent on skill-related activities, with 
the major emphasis on study skills or testing activities via workbooks or work- 
sheets. Students spent time with text, mostly the basal text, and most of the focus 
was on after reading activities. The greatest proportion of time was spent with 
materials at the word level. Students spent at least a third of their time listening, 
followed by transcribing and then silent reading. 

The frequent experiences of students with basal materials was an expected 
finding in the remedial inclass setting since one of the reasons for moving to this 
model of Chapter I service was to promote congruence between the remedial and 
developmental programs. Moreover, the finding that students had more individual 
contacts with teachers was also expected, given that the reading specialists began 
working with students after the developmental teachers had completed their group 
lessons. Thus, reading specialists in inclass settings often functioned as a monitor 
as students attempted to complete an assignment. However, the large amount of 
time in which students were not actually working (scheduled instruction ceased, 
no interaction, transition) was not expected; nor was the finding that students spent 
a large percentage of remedial reading time on skill work in workbooks or with 
worksheets. Also, the great variability in the experiences of students in the inclass 
setting speaks to the differences in how this instructional program was implemented 
across classrooms. 

As mentioned previously, much of the instruction received by students in these 
inclass remedial settings was focused on the basal materials that were used as part 
of the developmental or classroom reading program. In some instances, observers 
noted that these Chapter I students experienced a great deal of difficulty reading 
the basal textbook, but given the nature of the program, the reading specialists 
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continued to work with these materials, even though students may have been work- 
ing at a high error rate. The reading specialist, in those instances, tended to give 
more support by providing additional clues to help students complete assignments 
correctly. 

The nature of instruction experienced by students in these inclass settings 
reflected a change in role for the reading specialist. The greater percentage of time 
spent with the basal was an attempt to provide the congruence designated as impor- 
tant for students receiving instruction from two teachers (Allington, 1986; Johnston 
et al., 1985). Moreover, the reading specialists in these inclass settings tended to 
serve in the role of tutor or monitor, reinforcing the efforts of the classroom teacher, 
and serving in what Fraatz (1987) calls a supportive role. The variability in all three 
dimensions studied (teacher behaviors, lesson, and student behaviors) reflected the 
attempts of these reading specialists in the inclass programs to find their own way 
and to develop their own style of working in the same classroom as the student’s 
developmental reading teacher. 

One final point relates to the amount of time spent in remedial reading. Stu- 
dents in the inclass setting received an average of 310 minutes a week of classroom 
reading instruction and 50 minutes a week of remedial instruction (360 minutes of 
total reading instruction w'ith 50 minutes of it taught by a reading specialist), 
whereas students in the pullout setting received reading from their classroom 
teacher an average of 360 minutes a week plus 100 minutes of remedial reading 
when they were pulled for Chapter I services from a subject other than reading. 
Thus, students in the inclass setting received less remedial reading time and less 
classroom reading time than did students in the pullout group. 

Generalizabiliry issues. This study was conducted in a large urban district with 
Chapter I options similar to those available in other urban districts across our 
nation. Although the inclass model, with its 20-minute observation time for the 
remedial reading teacher, may differ slightly from models used in other districts, 
the compensatory programs (inclass and pullout) in this study are representative of 
compenstory programs in other districts in several ways. First, in any inclass model, 
reading specialists have to accommodate to the scheduling and approaches of the 
developmental or classroom teacher. Second, inclass remedial reading teachers 
work with groups of students who have shared a common experience with their 
classroom teacher, while remedial reading teachers in a pullout setting generally 
work with students who come from several different classrooms and, therefore, 
may have had different instructional experiences. Third, the expectations of this 
district, relative to the compensatory reading program; address the goals and objec- 
tives expected for most Chapter I reading programs, that is, students should be 
provided with instruction that helps them improve their basic skills and instruction 
should focus on helping students achieve success in their classroom reading 
program. 
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Summary and Implications 

Although significant differences were revealed on various dimensions of 
teacher behavior, lesson focus, and student behavior across settings, reading in- 
struction was generally viewed in both models as a series of discrete skill or basal 
lessons culminating in assessment of mastery on identified skills. However, pullout 
and inclass settings created different roles for the reading specialist and somewhat 
different experiences for students. Moreover, the implementation of the inclass 
model was quite idiosyncratic in terms of how time would be spent; there was 
much more variability across the three dimensions in what students experienced in 
remedial reading. Finally, although the inclass model did not correct some of the 
perceived problems associated with pullout settings, students in inclass models 
used more congruent materials in their remedial reading programs. Further, they 
spent more, not less, time in transition than did students in pullout remedial set- 
tings. Although setting has yet to be established as a factor influencing achievement 
of students, setting as it was implemented in this district had an impact on how 
teachers organized lessons, on what students experienced, and on the behaviors of 
the students within that setting. 

The movement across the country for greater integration of students into regu- 
lar education programs speaks to the need for a new vision for compensatory 
programs. As school district personnel search for means by which they can achieve 
this vision, the solution of changing setting (from pullout to inclass) appears to be 
much more complex and. perhaps less effective than one would hope. There are 
two points that need to be made regarding the findings of this study. First, setting 
can make a difference in terms of what students experience in compensatory educa- 
tion and that difference is not necessarily in the desired direction (e.g., increase in 
noninstructional time, focus on isolated skill practice). There is a continued need to 
study the nature of the instruction in both inclass and pullout settings. Specifically, 
researchers should move beyond the analysis of time allocated to various activities 
to an investigation of the nature of the instructional activities in a rich, qualitative 
manner (e.g., what were the explanations of the teachers in the instructional interac- 
tions? What sorts of activities were going on prior to reading? In what ways were 
students actively involved in the lesson?). Studies of instruction and explanation 
should be conducted in various compensatory education settings and at various 
levels of student ability/grade so that questions regarding the effects of these vari- 
ables can be addressed. 

Second, inclass models are not easy for remedial reading teachers to imple- 
ment. The variability in the functioning of the reading specialists in this study 
speaks to their struggle to fit into the classroom regimen. Given the emphasis on 
integration and the movement to alternative models for compensatory programs, it 
is imperative that reading specialists be given training to help them function effec- 
tively. Moreover, questions regarding the appropriate role for the reading specialist 
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need to be included in educational agendas. Although there is much enthusiasm 
for the “ideology of supportiveness “ (Fraatz. 1987, p. 70) from those who believe 
that the strongest compensatory program is one that provides for congruence or 
alignment, there is still much to be learned about the appropriateness and effective- 
ness of this notion as well as how it can be best implemented in the total school 
reading program. 
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Individual Differences in Reprocessing of Text 
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This study examined the roles of basic reading processes and prior knowledge in reprocessing of 
expository text. According to their performance in a decoding and a working memory task, 48 
college students were divided into groups of above-average and average readers and either rewrote 
notes, reread notes, or reread a text. The 3 strategies were equally effective in improving 
comprehension for text-explicit and text-implicit information, and reading ability and prior 
knowledge were more predictive for comprehension than was the type of reprocessing activity. 
Text reprocessing might help average readers to compensate for their lower performance in 
answering text-implicit questions, whereas above-average readers seem to combine more text 
information with their knowledge base. Working memory played the major role for compre- 
hending text-implicit information, whereas knowledge was relatively more important for explicit 
and script-implicit information. 



Reviews of note-taking research suggest that the process of 
taking notes does not globally facilitate comprehension and 
retention of discourse information (Hartley, 1983; Kiewra, 
1985, 1989). Evidence demonstrating that note-taking im- 
proves recall of important lecture information (Einstein, Mor- 
ris, & Smith, 1985), generation of connections between new 
information and prior knowledge (Peper & Mayer, 1986), and 
organization of coherent discourse representations (Cook & 
Mayer, 1983), supports the view that taking notes stimulates 
qualitatively different types of encoding activities rather than 
facilitating overall retention (Bretzing & Kulhavy, 1981; Hidi 
& Klaiman, 1983; Mayer, 1983; Smith & Tompkins, 1988). 

The majority of studies, however, have concluded that the 
primary value of taking lecture notes is not to facilitate 
encoding but to improve comprehension by providing stu- 
dents with an external storage available for reprocessing 
(Carter & Van Matre, 1975; Ladas, 1980; Rickards & Fried- 
man, 1978; Spires & Stone, 1989). These studies suggest that 
information is more likely to be remembered when the infor- 
mation is recorded in notes and reviewed. 

AJthough there is considerable evidence demonstrating the 
importance of reviewing notes in learning from lectures, the 
relative effectiveness of different review techniques remains 
to be specified (Barnett, Di Vesta, & Rogozinski, 1981; Kie- 
wra, 1983; Kiewra, DuBois, Christian, & McShane, 1988) 
and extended to the more limited research on reprocessing of 
written texts. 

In investigating review effects on learning from expository 
text, Crewe (1969) found that reviewing a previously under- 
lined text increased retention as compared with a no-review 
condition. More recently Kardash and Kroeker (1989) ex- 
amined the comprehension performance of students who 
reviewed and revised their notes, varying the timing of the 
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review period. Subjects who reviewed their notes prior to the 
test (2 days after acquisition) recalled more idea units than 
subjects who reviewed notes immediately after acquisition, 
subjects who reviewed notes 1 day after acquisition, subjects 
who did not review, and subjects who reviewed mentally prior 
to the test (2 days after acquisition). This effect was attenuated 
when cloze and application test measures were analyzed. 
Kardash and Kroeker (1989) suggested that reviewing notes 
prior to the time of testing facilitates comprehension when 
the test provides no retrieval cues, whereas the activity of 
review might not be beneficial when a test itself provides 
retrieval cues. 

Although there is some evidence that students who take 
notes during reading recall more text information (Bretzing 
& Kulhavy, 1979; Shimmerlick & Nolan, 1976), studies com- 
paring the effects of different study techniques on reading 
comprehension have found simpler activities, such as under- 
lining or repeated reading, to be at least as effective as note- 
taking. For example, a study by Jonassen (1984) revealed that 
readers who were instructed to interrelate important ideas 
during note-taking recalled more inforfinrtion of structural 
importance on a delayed test in comparison with readers who 
took notes with no integrative instruction, readers who under- 
lined important ideas, and readers who just read the text. 
Similarly, in his review of study strategy research, Anderson 
(1980) found only two studies that reported a rereading strat- 
egy inferior to either note-taking or underlining, whereas 
several studies revealed no comprehension difference between 
repetitive reading, note-taking, and underlining. An analysis 
of Arnold’s (1942) data showed that a rereading strategy 
improved comprehension on immediate and delayed tests in 
comparison with note-taking, outlining, and summarizing 
(Anderson, 1980). Consistent with this result, Howe and 
Singer (1975) showed that students who reread unfamiliar 
texts performed better on immediate and delayed free recall 
than subjects who summarized or copied the texts verbally, 
and Kardash, Amlund, and Kulhavy (1984) found no signif- 
icant differences in recall when subjects were required to 
follow either a rereading, a note-taking, or an underlining 
strategy, with processing time held constant across conditions. 
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TEXT REPROCESSING 



Recently, researchers have examined repetitive reading 
more closely as a text reprocessing strategy rather than as a 
control condition for other techniques such as note-taking 
(e.g., Dowhower, 1989). Consistent with earlier results re- 
ported by Rothkopf (1968), Amlund, Kardash, and Kulhavy 
(1986) found that subjects who read a text twice recalled 
significantly more information in comparison with subjects 
who read the text once or three times. In their attempt to 
qualify the effects of rereading on learning from expository 
text, Amlund et al. (1986) further suggested that rereading 
primarily facilitates retention of main ideas and that persist- 
ence of errors in recall increases with repeated reading. In- 
cluding ability (average College Board Achievement Test 
scores) as an additional factor, Barnett and Seefeldt (1989) 
found that repeated reading increased factual retention for all 
students, whereas only high-ability students improved their 
performance on a transfer task after a second reading trial. 
This result suggests that the effectiveness of different reproc- 
essing strategies for comprehension are partly determined by 
individual differences in information-processing abilities. 

As Kiewra (1989) has pointed out, only a few studies have 
investigated the ways in which cognitive differences among 
learners affect note-taking during lectures, and the role of 
such differences in taking notes from written texts and review 
activities has been essentially ignored. Earlier studies (Ber- 
liner, 1969; Di Vesta & Gray, 1973) showed that only those 
subjects with a greater ability to hold digit strings in working 
memory profited from note-taking, whereas note-taking in- 
terfered with comprehension for subjects who were low in 
working memory capacity. Extending these results, Kiewra 
and Benton (1988) found that subjects more able to hold and 
manipulate discourse information in working memory re- 
corded more words, complex propositions, and main ideas 
than did subjects with lower ability. As these factors were 
positively related to lecture comprehension, this result indi- 
cates that subjects high in working memory capacity profited 
more from taking notes. 

Other studies focusing on individual differences have found 
prior knowledge to affect note-taking behavior. For example, 
Peper and Mayer (1978, 1986) demonstrated that notetakers 
who were low in prior knowledge about the lecture topic 
performed better on problem-solving tasks and worse on 
factual retention and recognition tasks, whereas taking notes 
did not improve problem-solving performance for students 
with adequate prior knowledge. Peper and Mayer(1978, 1986) 
suggested that note-taking is most likely to facilitate encoding 
of information that is low in familiarity. 

In summary, there is some evidence showing that taking 
notes during reading may improve recall of text information, 
but the primary value of notes seems to be their availability 
for review prior to comprehension tests. On the other hand, 
the activity of rereading text has been found to be as beneficial 
to comprehension as taking or reviewing notes. Other results 
also suggest that individual differences in cognitive processing 
may affect reprocessing behavior, but there has been little 
effort to systematically examine the relative effects of reading 
comprehension components on different reprocessing strate- 
gies of text. 

In this study, we investigate decoding ability, working mem- 



ory, and domain-specific prior knowledge as predictors of 
reading comprehension after either rewriting notes, rereading 
notes, or rereading a text. Through this investigation we 
attempt to specify the relative contributions of reading com- 
ponents to different reprocessing activities. 

The study is based on a framework of reading comprehen- 
sion that combines an information-processing account with a 
schema-based or knowledge-based account. According to this 
view, which was developed by Perfetti (1988, 1989). compre- 
hension is determined primarily by the efficiency with which 
persons use local language processes such as word identifica- 
tion and encoding of propositions in working memory (Dane- 
man & Carpenter, 1980; Kintsch & Van Dijk, 1978; Lesgold, 
Roth, & Curtis, 1979). To the extent that word decoding is 
resource demanding, assembly and integration of propositions 
in working memory are less efficient. As a consequence, the 
quality of temporarily constructed text representations suffers 
and affects, along with a reader’s knowledge base, the con- 
struction of a text model (Anderson, Spiro, & Anderson, 
1978; Johnson-Laird, 1983; Just & Carpenter, 1987; van Dijk 
& Kintsch, 1983). 

A convergence of results from studies investigating reading 
comprehension within an information-processing framework 
has found general language comprehension processes, meas- 
ured by listening comprehension (Palmer, MacLeod, Hunt, 
& Davidson, 1985), word decoding (Frederikscn, 1982; Oak- 
hill & Gamham. 1988), and working memory constraints 
(Baddeley, Logie, Nimmo-Smith. & Brereton, 1985; Cun- 
ningham. Stanovich, & Wilson. 1990: Goldman, Hogaboam, 
Bell, & Perfetti, 1980; Rapala & Brady, 1990) to contribute 
independently to reading comprehension. Consistent with a 
schema-based theory of reading comprehension (Wilson & 
Anderson, 1986), there is ample evidence demonstrating the 
benefits of a reader’s prior knowledge source on memory and 
comprehension (Anderson & Acker, 1984; AfTlerbach, 1990; 
Reutzel & Morgan, 1990; Spilich, Vesonder, Chiesi, & Voss, 
1979). 

Recent studies including analysis of prior knowledge, as 
well as tests of reading comprehension ability, suggest that 
specific knowledge allows poor readers to .compensate for 
their generally low comprehension ability in a familiar text 
domain (Adams, Bell, & Perfetti, 1990; Recht & Leslie, 1988; 
Yekovich, Walker, Ogle, & Thompson, 1990). In a previous 
attempt to examine the relative contributions of specific in- 
formation-processing correlates of reading and prior knowl- 
edge to college-level reading comprehension, Haenggi and 
Perfetti (1991) identified word and pscudoword decoding, 
working memory, and knowledge as significant predictors of 
expository text comprehension. Furthermore, prior knowl- 
edge was more important for comprehension of information 
stated explicitly in the text, whereas working memory capacity 
made a larger contribution to comprehending information 
that had to be inferred from the text. 

To specify the role of individual differences in reprocessing 
of text, we take into account the relative contributions of 
basic reading processes and knowledge to comprehension. 
Given the facilitating effects of different reprocessing strategies 
on reading comprehension, we expected that either rewriting 
notes, rereading notes, or rereading a text would generally 
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improve comprehension. Assuming the crucial roles of prior 
knowledge and basic reading processes in comprehension, we 
also expected large roles for knowledge, decoding, and work- 
ing memory. Determining the relative effectiveness of specific 
reprocessing activities and individual differences in reading 
ability, as well as in prior knowledge, should provide a further 
qualification of students’ reading comprehension behavior, 
which is a prerequisite to improving instruction. 

Method 

Subjects 

Forty-eight university undergraduates fulfilling a course require- 
ment served as subjects. We used a median split to divide .subjects 
into above-average and average readers on the basis of a composite 
score of reading ability. This measure combines reading speed with 
accuracy parameters and is an extension of a reading-efficiency 
measure proposed by Jackson and McClelland (1979). A measure of 
reading ability was calculated by multiplying the percentage of cor- 
rectly recalled target words by the number of syllables read per minute 
in a probe discourse task and dividing this product by average 
vocalization speed for words and pseudowords. The mean reading 
scores (range = 60 to 427) for above-average and average readers, 
respectively, were 245 ( SD = 58) and 105 ( SD *= 33), and subjects in 
the three reprocessing conditions were matched according to their 
reading ability scores. Because of the multiplicative measurement of 
reading ability, some above-average readers might have scored higher 
because they sacrificed word-recall accuracy for reading speed. How- 
ever, three separate analyses of variance on the recall and raw reading- 
speed data showed that above-average readers recalled more words in 
the probe discourse task (A/ = 62.7%, SD » 10%) than average 
readers (M - 56.2%, SD = 8.6%), F[ 1, 46) = 23.12, p < .001. In 
addition above-average readers showed shorter reading times for 
sentences in the probe discourse task (M = 252 syllables per min, SD 
= 55), F\ 1 , 46) « 35.0 1 , p < .001, and shorter vocalization latencies 
in the decoding task ( M = 643 ms, SD =* 65 ras), F\ 1 , 46) ® 1 3.69, p 
<.001, In comparison, average readers read 162 syllables per min 
(SD = 50) in the probe discourse task and vocalized words and 
pseudowords with a mean latency of 774 msec (SD «= 160 msec). 

Design 

Reading ability was used as an independent grouping variable. As 
a second factor, reprocessing type (rewrite notes vs. reread notes vs. 
reread text) was varied among subjects. Learning trial (processing the 
text in Session 1 vs. Reprocessing the text in Session 2) and question 
type (text-explicit vs. text-implicit vs. script- implicit) served as within- 
subject factors. The design was a 2 (reading ability) x 3 (reprocessing 
type) x 2 (learning trial) x 3 (question type) analysis of variance, 
with repeated measures on the learning trial and question type factors. 
Prior knowledge was included as a covariate. Accuracy in responding 
to multiple-choice comprehension questions and sentence verifica- 
tions, as well as verification latencies, were the dependent measures. 

Materials and Procedure 

The study was conducted in two sessions, the first lasting about 2 
hr and the second lasting about 1 hr. Subjects were to perform 
individually the vocalization task, the probe-discourse task, and the 
prior knowledge test and to carry out the reading task, as well as a 



sentence-verification and a multiple-choice comprehension test. Sub- 
jects were told either to take notes during reading or to read the text 
without taking notes. One day later, subjects were asked to rewrite 
their notes, to reread their notes, or to reread the text. After the 
reprocessing trial, the subjects were asked to complete a second set of 
sentence-verification and multiple-choice tests. The vocalization, the 
probe-discourse, and the two sentence- verification tasks were imple- 
mented with Micro Experimental Laboratory (Schneider, 1988), and 
a personal computer (Toshiba T 1 600) was used to present the stimuli. 
Vocalization and probe-recall latencies were measured with a re- 
. sponse box with voice key. 

Word and pseudoword vocalization task. Subjects read and vo- 
calized 40 high-frequency (> 100 per million) words, 40 low- frequency 
(1-7 per million; Meier, 1978) words, and 40 pronounceable pseu- 
dowords. For each word type, there were 10 items of the following 
length categories: five-letter/one-syllable words, five-let ter/two-sylla- 
ble words, eight-letter/two-syllable words, and eight-letter/three-syl-. 
lable words. We matched the initial phoneme across the three word 
types because this factor can have an influence on vocalization latency 
(Frederiksen & Kroll, 1976). Word type and length were the within- 
subject variables, and decoding speed of correct responses was the 
dependent variable. 

The stimuli were presented on a computer screen, and between 
each trial a fixation dot was displayed. A first block of 80 words was 
presented first, followed by a second block of 40 pseudowords, and 
the four length categories were randomized within each block. Re- 
sponse latencies were measured from stimulus onset to onset of 
vocalization. 

Probe-discourse task. We used a probe -recall procedure to assess 
availability of recently encoded discourse segments in working mem- 
ory. Subjects were presented with a text (about human thinking) that 
was approximately 700 w ords in length and of graduate level (Level 
17) in readability (Fry, 1977). Single words from the text segment 
just read reappeared as probes, and subjects were asked to recall the 
target word that had followed a specific probe. The text segments 
varied in length (probe distance) and in occurrence versus nonoc- 
currence of a sentence boundary. For short segments (near probes), 
2 content words, or between 2 and 7 content and functional words, 
intervened between the initial occurrence of the probe word and the 
recall test. For long segments or far probes, the probe test occurred 
after 5 content words, or between 7 and 12 words total. In a current 
text segment, the probe occurred in the sentence being read at the 
time of the test. In a preceding text segment, probe words occurred 
in the sentence preceding the one being read at the time of probe 
recall. Probe distance and segment type were the within-subject 
variables, and response accuracy was the dependent variable of inter- 
est. Reading times for each line of text were additionally recorded. 
Each Probe Distance x Segment Type combination occurred eight 
times in the text. 

Subjects silently read one line of text at a time from the screen. 
Pressing a button on the response box permitted the subject to obtain 
either the next line or a probe word. As soon as the subject vocalized 
the word that followed the probe in the preceding text segment, the 
probe was replaced with the next line of text 

The probe-discourse task was found to be more highly correlated 
with comprehension measures such as the Davis Reading Test (Davis 
& Davis, 1962), r = ,68, and the Jackson and McClelland (1979) 
reading comprehension test, r = ,78, than was the Daneman and 
Carpenter (1980) reading span task, rs *= .49 and .53 (Hammond, 
1987), These data indicate that the probe-discourse task might mea- 
sure components that are more similar to those involved in reading 
comprehension than those involved in the more frequently used 
reading span task. 

Prior knowledge test. Twenty open-ended questions covered basic 
concepts in human problem-solving and decision-making theories. 
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Subjects were allocated 20 min to respond to short-answer informa- 
tion (e.g., “How can you support a decision-making process?”); the 
number of correctly answered questions was registered. 

Reading task. Subjects were presented with a text 8 pages in 
length describing three different approaches to explaining human 
decision making (i.e., normative rules, heuristics, and problem-solv- 
ing approach). The construction of the text was based on Huber 
(1982). A content analysis divided the text into 41 thematic blocks, 
or macropropositions (Meyer, 1985), and these were subdivided into 
230 smaller idea units. This analysis served as a guideline for con- 
structing sente nce-verificati on and multiple-choice comprehension 
tests. Subjects were instructed to read the text while either taking 
notes or not taking notes. In either case students were informed that 
they would have to answer questions after the completion of the 
reading and that they would be given the opportunity to reprocess 
the text 1 day later. 

Subjects were asked 24 hr later to rewrite their notes, to reread the 
notes, or to reread the text. The text was available for all subjects 
during reprocessing. The two note-taking groups were instructed to 
focus on their notes during reprocessing; post-experimental question- 
ing confirmed that no subject in these groups simply reread the text. 
To hold learning time constant across the three reprocessing groups, 
original processing time, as well as reprocessing time, was recorded 
for those 16 subjects who rewrote their notes. Subjects who were 
asked either to reread their notes or to reread the text were matched 
according to these processing times. As a result of this yoking proce- 
dure, each experimental group spent an average of 38 min ( SD « 4.7 
min) processing the text on the first trial and 35 min (SD » 4.3 min) 
reprocessing the text. 

Reading comprehension tests. After each text-processing trial, 
subjects were confronted with two different versions of a sentence- 
verification and a multiple-choice comprehension test. We based the 
construction of these tests on 80 statements that were either explicit 
or implicit with respect to information covered by the text. Forty 
explicit statements covered information directly stated in one sen- 
tence of the text. Forty implicit statements required the reader either 
to combine information from two sentences that appeared separated 
in the text or to draw inferences from his or her prior knowledge 
base. These categories are referred to as text-implicit and script- 
implicit statements (Pearson & Johnson, 1978), respectively, and 
both were represented by 20 items. With an interTater agreement of 
95%, two independent raters allocated the 80 statements to the three 
defined categories before the items were split into two sets of 40 
statements. Then we constructed sentence-verification and multiple- 
choice test versions for each set. After completing the first processing 
trial on the text, subjects received one version of a sentence-verifica- 
tion test and a multiple-choice test; the other version of the corre- 
sponding tests was completed following the reprocessing trial. Thus, 
subjects were presented with each test item in two different modes 
(sentence verification or multiple choice), and the presentation order 
of test versions was randomized across the three reprocessing condi- 
tions. 

After reading the text, all subjects were shown 40 statements to 
which they were to respond true or false as rapidly as possible. To 
control for information quantity, an average length of 104 words per 
sentence was held constant across the explicit, text-implicit, and 
script-implicit sentence types. One half of the statements were false 
and the other half were true. The dependent measures were verifica- 
tion time and accuracy. Each sentence was presented in random order 
on the computer screen, and subjects used the index and middle 
fingers of their preferred hand to press the assigned “yes” or “no" 
keys. The verification task was followed by 40 multiple-choice ques- 
tions. Items were randomized, and presentation order was the same 
for all subjects. The procedure for measuring comprehension was the 
same after the reprocessing trial, except that subjects received multi- 



ple-choice questions from Session 1 as sentence verifications and vice 
versa. 

Results 

As was pointed out, the decoding and working memory 
tasks were used to form a composite measure of reading 
ability (see the Appendix for individual analyses of variance 
of these tasks). For subjects who rewrote or reread their notes, 
an analysis showed that the two reader groups did not differ 
in the quantity of notes taken. Above-average readers did not 
take significantly more notes (M = 53 idea units, SD = 13) 
than average readers (M = 49 idea units, SD = 13), |r| < 1, 
when they read the text the first time. Rewritten notes con- 
tained an average number of 50 idea units (SD = 14) for 
above-average readers and 48 idea units (SD ~ 19) for average 
readers, |/| < 1. An analysis of note- taking quality focused on 
the occurrence of elaborative processes (e.g., inferences, gen- 
eralizations of idea units) in notes. The mean frequency of 
elaborations was generally low (< 1). indicating that readers 
took primarily verbatim notes. 

To examine the effects of the three reprocessing conditions 
on comprehension relative to reading ability and prior knowl- 
edge, we first report analyses of variance. Then, to investigate 
the relative contributions to reading comprehension of decod- 
ing speed, working memory accuracy, and knowledge, we 
report the results of correlations and stepwise regressions. 

Reprocessing Effects Relative to Reading Ability and 
Prior Knowledge 

First, the results were analyzed by performing three separate 
2 x 3 x 2 x 3 analyses of covariance, one with multiple- 
choice test accuracy as the dependent variable, one with 
sentence- verification accuracy as the dependent measure, and 
one with sente nce-verificati on latencies as the dependent vari- 
able. In each analysis, reading ability (above-average vs. av- 
erage) and reprocessing type (rewrite notes vs. reread notes 
vs. reread text) were between -subjects factorsr whereas learn- 
ing trial (1 vs. 2) and question type (explicit vs. text-implicit 
vs. script-implicit) served as within-subject factors. Further- 
more, we included prior knowledge as a covariate for all 
between-subjects analyses. The significance level was fixed at 
an alpha of .05. 

A learning trial effect, F( 1 , 42) = 39.05, p < .001 , revealed 
a higher portion of correct multiple-choice answers after a 
reprocessing trial (M = 66%, SD = 15%) than after a first 
learning trial (M = 59%, SD = 1 6%). 

A question type effect, P( 2, 84) * 152.32, p < .001, indi- 
cated a higher response accuracy for explicit questions (M = 
73%, SD = 1 1%) in comparison with text-implicit questions 
(M = 57%, SD = 11%), f(84) = 11.17, p< . 001, and script- 
implicit questions (M « 48%, SD = 11%) were answered less 
accurately than text-implicit questions, f(84) - 6.28, p < .001 . 

Above-average readers answered more multiple-choice 
questions correctly (M = 66%, SD = 16%) than average 
readers (M = 59%, SD = 14%), fll, 41) = 6.55, p < .02. 
Table 1 includes the mean multiple-choice accuracies for the 
two reader groups. 
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Table 1 

Mean Percentages and Standard Deviations of Correctly Answered Explicit (Exp). 
Text-Implicit (T-Imp) and Script-Implicit (S-Imp) Multiple-Choice Questions 
for Above-Average and Average Readers on Learning Trials I and II in Three 
Reprocessing Conditions 







Learning Trial 1 






Learning Trial 11 




Group/ 

reprocessing condition 


Exp 


T-Imp 


S-Imp 


Exp 


T-Imp 


S-lmp 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


Above-average readers 


Rewrite notes 


76 


12 


56 


17 


52 


9 


84 


11 


64 


15 


53 


1U 


Reread notes 


68 


14 


56 


13 


46 


13 


83 


11 


58 


13 


53 


15 


Reread text 


71 


6 


58 


7 


51 


12 


81 


10 


68 


12 


56 


9 


Average readers 


Rewrite notes 


64 


7 


49 


8 


45 


9 


64 


11 


59 


16 


41 


8 


Reread notes 


66 


13 


49 


8 


46 


13 


77 


8 


63 


13 


46 


14 


Reread text 


65 


13 


45 


15 


45 


8 


74 


13 


61 


14 


43 


17 



Note. Values are given in percentages. 



Reprocessing type did not affect multiple-choice accuracy, 
p < 1 1 but prior knowledge made a significant contribution, 
F\ i, 41) « 24.64, p < .001. Learning trial interacted with 
question type, F\ 2, 84) = 10.77, p < .001, indicating that 
explicit questions were answered more accurately after the 
second learning trial (M ** 77%, SD *=» 12%) than after the 
first learning trial (M =* 68%, SD » 1 1%), /(84) =» 8.49, p < 
.00 1 . Higher accuracy rates were shown for text-implicit ques- 
tions when they were answered after reprocessing the text (A/ 
s 62%, SD « 13) than when they were answered during the 
first trial ( M m 52%, SD - 12%), f(84) *= 9.43, p < .001. In 
contrast, script-implicit questions remained at the same ac- 
curacy level from the first trial (A/ = 48%, SD = 1 1 %) to the 
second trial (M = 48%, SD = 1 3%), |/| < 1 . 

A Learning Trial x Question Type x Ability interaction, 
F{2 t 84) = 5.54, p « .005, revealed the following group 
differences: Average readers showed relatively greater im- 
provement from the first to the second learning trial on text- 
implicit questions, f(84) = 3.51, p < .001, whereas above- 
average readers showed relatively greater improvement on 
script-implicit questions, t{ 84) - 2.65, p < .01. Improvement 
scores for explicit questions were only marginally higher for 
above-average readers, f( 84) « 1.85, p < .10. For these group 
comparisons, we used a covariate adjustment of the mean 
multiple-choice accuracy improvement scores. The adjusted 
scores for above-average readers were 10.7%, 6.1%, and 3.6% 
for explicit, text-implicit, and script-implicit questions, re- 
spectively. The corresponding improvement scores for aver- 
age readers were 6.8%, 13.5%, and —2.0%. 

A second four-factor analysis of covariance including prior 
knowledge as a co variate for the between-subjects effects was 
based on sentence-verification accuracy as a dependent vari- 
able. Sentences were correctly verified at a higher rate when 
the test was taken after reprocessing (M = 70%, SD « 12%) 
than after a first reading trial (M = 61%, SD =* 11%), FU, 
42) 13 78.25, p < .001. 

A question type effect, F\2 t 84) = 40. 1 1 , p < .00 1 , reflected 
lower verification accuracies for text-implicit sentences (A/ = 



62%, SD - 12%) than for explicit sentences (M = 72%, SD 
= 7%), /( 84) = 5.82, p < .001, and text-implicit sentences 
were verified more accurately than were script-implicit sen- 
tences (M = 57%, SD = 12%), f(84) - 2.91, p< .01. 

Above-average readers verified the sentences more accu- 
rately (A/ = 70%, SD = 11%) than did average readers (M = 
61%, SD « 12%), F\\, 41) = 20.56, p < .001. Sentence- 
verification accuracy data for the two groups are summarized 
in Table 2. Verification accuracy was not dependent on 
reprocessing type, F < 1, and prior knowledge contributed 
again to comprehension performance, F\\ t 41) = 16.88, p < 
.001. 

A Learning Trial x Question Type interaction, F{ 2, 84) « 
3.77, p < .03, indicated relatively less improvement from the 
first to the second learning trial on script-implicit sentences 
(M - 5.1%, SD = 14%) in comparison with both explicit 
sentences (A/ = 9.0%, SD = 9%), /(84) = 2.32, p < .05, and 
text-implicit sentences ( M = 1 1 .8%. SD = 1 2%), t( 84) = 4.06, 
p < .001, whereas improvement scorertxi explicit and text- 
implicit sentences did not differ significantly, t( 84) = 1.62, p 
> .05. 

A third 2 x 3 x 2 x 3 analysis of covariance, which included 
sente nee- verification times as dependent measures, revealed 
only two main effects. Sentences were verified faster after the 
second learning trial (6,841 msec, SD = 1,409 msec) than 
after the first learning trial (7,429 msec, SD = 1,443 msec), 
F{\ % 42) = 13.24, p<. 001. 

A question type effect, F\2, M ) » 79.97, p < .001, indicated 
shorter verification times for explicit sentences (6,679 msec, 
SD - 1,245 msec) than for text-implicit sentences (7,231 
msec, SD = 1 ,390 msec), /(84) = 4.34, p < 0.001 , and response 
times were longer for script-implicit sentences (8,263 msec, 
SD = 1,722 msec) than for text-implicit sentences, f( 84) = 

8.12, p < .001. 

Sentence- verification times did not significantly differen- 
tiate above-average readers (A/ = 6,831 msec, SD = 1,127 
msec) from average readers (7,439 msec, SD = 1,427), FT(1, 
41) = 1.61, p > 0.2, and neither reprocessing type nor prior 
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Table 2 



Mean Percentages and Standard Deviations of Correctly Verified Explicit (Exp). 
Text-Implicit (T-Imp). and Script-Implicit (S-Imp) Sentences for Above-Average and 
Average Readers on Learning Trials I and II in Three Reprocessing Conditions 



Group/ 

reprocessing condition 




Learning Trial 1 






Learning Trial 11 




Exp 


T-Imp 


S-lmp 


Exp 


T-lmp 


S-lmp 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


Above-average readers 


























Rewrite notes 


76 


6 


59 


16 


61 


15 


82 


10 


72 


19 


64 


10 


Reread notes 


69 


7 


60 


14 


63 


16 


77 


8 


70 


8 


63 


15 


Reread text 


69 


7 


63 


9 


58 


15 


79 


6 


79 


10 


64 


7 


Average readers 


























Rewrite notes 


64 


7 


46 


13 


49 


10 


69 


5 


61 


11 


50 


12 


Reread notes 


63 


6 


55 


5 


49 


18 


81 


11 


69 


15 


56 


14 


Reread text 


64 


10 


53 


13 


48 


7 


71 


6 


55 


12 


60 


15 



Note Values are given in percentages. 



knowledge affected verification times, both Fs< The mean 
verification times for the two reader groups are displayed in 
Table 3, 

Contributions of Decoding , Working Memory , and 
Prior Knowledge to Reading Comprehension 

We examined the relative contributions of decoding, work- 
ing memory, and prior knowledge to comprehension of ex- 
plicit, text-implicit, and script-implicit information by step- 
wise regressions. On the basis of a framework of reading 
(Perfetti, 1988), we entered the measures according to their 
assumed processing complexity. To predict reading compre- 
hension accuracy, as well as sentence-verification time, mean 
correct vocalization latencies for words and pseudowords, 
probe-recall accuracy, and prior knowledge were entered in 
three steps. To form a mean comprehension accuracy score, 
we combined multiple-choice and sentence-verification ac- 
curacies. These two measures were highly correlated for both 
above-average readers, r » .84, df° 22, p < .001 , and average 
readers, r= .78, df = 22, p< .001. 



Correlations between reading-comprehension accuracy and 
the three predictors are presented in Table 4. Probe-discourse 
accuracy was moderately related to comprehension perform- 
ance for text-implicit and explicit information (with 46 df r 
> .30 is significant at p < .05). Prior knowledge also was 
moderately related to accuracy scores for explicit and script- 
implicit questions. Furthermore, accuracy scores for explicit, 
text-implicit, and script-implicit questions that were taken 
after the first learning phase substantially correlated with the 
corresponding scores following a reprocessing trial (rs « .73, 
.77, and .66, respectively). 

We performed three stepwise regressions using comprehen- 
sion accuracy for each question type as a separate dependent 
variable. The results of these regressions are sum- diarized in 
Table 5. 

Entering the three predictors into a regression equation to 
predict comprehension performance for explicit questions, we 
found 67.3% of the variance explained. (Correcting for shrink- 
age produced an adjusted R 2 of .65). Table 5 shows that 
probe-discourse accuracy (35%, p < .001) and prior knowl- 
edge (28.3%, p < .001) accounted for significant proportions 
of variance in comprehension for explicit questions. 



Table 3 



Mean Response Times and Standard Deviations in Milliseconds for Explicit (Exp). Text-Implicit (T-Imp), and 
Script-Implicit (S-Imp) Sentence Verifications for Above-Average and Average Readers on Learning Trials I and II in 
Three Reprocessing Conditions 









Learning Trial 1 










Learning Trial 11 






Group/ 


Exp 


T-lmp 


S-lmp 


Exp 


T-Imp 


S-lmp 


reprocessing condition 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


Above-average readers 
Rewrite notes 


6,432 


819 


7,211 


1,409 


8,064 


1,456 


5,928 


1,344 


6,902 


1,832 


7,665 


1,661 


Reread notes 


6,394 


1,328 


6,968 


896 


8,201 


2,018 


5,962 


1,208 


6,019 


1,221 


8,015 


1,754 


Reread text 


6,991 


1,285 


7,307 


1,211 


8,201 


1,618 


6,489 


1,317 


6,945 


1,096 


8,023 


1,493 


Average readers 
Rewrite notes 


6,778 


1,001 


7,383 


1,069 


8,252 


1,067 


6,945 


1,684 


7,309 


2,002 


8,102 


1,999 


Reread notes 


8,031 


1,716 


8,518 


1,738 


9,093 


1,905 


6,568 


587 


7,195 


690 


8,688 


1,088 


Reread text 


7,195 


1,848 


8,020 


3,117 


9,266 


3,292 


6,446 


1,795 


6,997 


2,321 


7,585 


2,564 



Note. Only correct responses are included. 
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Correlations of Explicit (Exp). Text-Implicit (T-Imp). and Script-Implicit (S-Imp) 
Comprehension Accuracy Measures with Decoding Speed (Decoding). Probe Discourse 
Accuracy (Probe), and Prior Knowledge (Knowledge) for Learning Trials I and II 



Measure 


1 


2 


3 


4 


5 


6 


7 


8 


9 


1. Decoding 


— 


















2. Probe 


-.19 


— 
















3. Knowledge 


-.10 


.19 


— 














4. Exp I 


-.21 


.59* 


.61* 


— 












5. T-Imp 


-.04 


.62* 


.20 


.52* 


— 










6. S-lmp 


-.23 


.42* 


.67* 


.52* 


.34* 


— 








7. Exp II 


-.16 


.56* 


.59* 


.73* 


.63* 


.60* 


— 






8. T-lmp 11 


-.04 


.61* 


.26 


.59* 


.77* 


.23 


.67* 


— 




9. S-Imp 11 


-.05 


.33* 


.71* 


.60* 


.43* 


.66* 


.62* 


.45* 


— 



Note. 1 = Learning Trial 1: 11 = Learning Trial 11. 
* p < .05, two-tailed. 



In predicting comprehension accuracy for text-implicit in- 
formation, the working memory variable made the only sig- 
nificant contribution (43.8%, p < .001). The three predictors 
together explained 45.6% of the variance in comprehension 
(adjusted R 2 « .418). 

When accuracy scores for answering script-implicit ques- 
tions served as a dependent variable, probe-discourse accuracy 
contributed significantly to the variance explained (15.2%, p 
< .01), and prior knowledge made the largest contribution 
(47%, p < .001). The three predictor variables explained 
64.4% of variance in comprehension performance for script- 
implicit questions (adjusted R 2 a .620). 

A second correlational analysis (see Table 6) showed that 
verification times for all three question types correlated with 

Table 5 

Summary of the Stepwise Prediction of Text Comprehension 
Performance for Explicit. Text-Implicit, and 
Script-Implicit Questions 



Question type/ 
variable 
entered 


R 


R l 


Change 
in R* 


Change 

inF 


Explicit 








1.89 


1. Decoding 
speed 

2. Probe discourse 
accuracy 


.199 


.040 


.040 


.624 


.390 


.350** 


25.80 


3. Domain-specific 
prior knowledge 
Text-implicit 


.820 


.673 


.283** 


38.06 


1. Decoding 
speed 


.045 


.002 


.002 


0.09 


2. Probe discourse 
accuracy 


.663 


.440 


.438** 


35.17 


3. Domain-specific 
prior knowledge 
Script-implicit 


.675 


.456 


.016 


1.25 


1. Decoding 
speed 


.147 


.022 


.022 


1.02 


2. Probe discourse 
accuracy 


.417 


.174 


.152* 


8.27 


3. Domain-specific 
prior knowledge 


.802 


.644 


.470** 


58.10 



*p<.01. **p< .001. 



word and pseudoword vocalization speed, whereas the time 
measures were not significantly related to probe discourse 
accuracy and prior knowledge. Verification times were also 
significantly related to each other on both learning trials, rs — 
.61, .40. and .71 for explicit, text-implicit, and script-implicit 
questions, respectively. 

To examine wether decoding, working memory, and prior 
knowledge contributed to sentence-verification times, we per- 
formed another set of three stepwise regressions. Table 7 
summarizes the results of these regressions. 

When the three predictors were entered into a regression 
analysis to predict verification times for explicit sentences, 
23.6% of the variance was explained (adjusted R 2 *= .184). 
With a significant portion of 21.3% ( p < .001), decoding 
speed accounted for most of the variance explained. 

Table 7 shows the same pattern of results for the two 
remaining question types. With regard to text-implicit sen- 
tences, decoding accounted for 17.7% {p < .001) of the 
variance in verification time. The total percentage of variance 
explained was only 19.4% (adjusted R 2 « .139). 

When verification time for script-implicit sentences was 
predicted, the decoding variable explained^) 7.5% ( p < 0.00 1 ), 
and the three predictors accounted for 18.1% of variance 
(adjusted R 2 * .125). 

Discussion 

Consistent with the hypothesis that the activity of reproc- 
essing text generally facilitates reading comprehension, more 
multiple-choice questions and sentence verifications were an- 
swered correctly and sentences were verified faster after a 
reprocessing trial. This result adds to other demonstrations 
that the activity of review or reprocessing improves compre- 
hension of both orally presented (e.g., Bromage & Mayer, 
1986; Knight & McKelvie, 1986) and written texts (e.g., 
Kardash & Kroeker, 1989). 

In comparison with earlier work in the field, different 
reprocessing activities in this study were systematically varied 
and related to individual differences in cognitive processing 
components of reading. We found that rewriting notes, re- 
reading notes, and rereading the text were equally effective 
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Table 6 

Correlations of the Explicit (Exp). Text-Implicit (T-Imp). and Script -Implicit (S-Imp) 
Sentence Verification Times with Decoding Speed (Decoding). Probe Discourse 



Accuracy (Probe), and Prior Knowledge (Knowledge) for Learning Trials I and II 



Measure 


1 


2 


3 


4 


5 


6 


7 


8 


9 


1. Decoding 


_ 


















2. Probe 


-.19 


— 
















3. Knowledge 


-.10 


.19 


— 














4. Exp l 


.48* 


-.07 


-.11 


— 












5. T-Imp I 


.34* 


-.20 


-.08 


.68* 


— 










6. S-Imp I 


.44* 


-.16 


-.17 


.80* 


.64* 


— 








7. Exp II 


.35* 


-.11 


-.08 


.61* 


.48* 


.66* 


— 






8. T-Imp II 


.37* 


-.12 


-.09 


.57* 


.40* 


.62* 


.84* 


— 




9. S-Imp II 


.33* 


.08 


.01 


.62* 


.61* 


.71* 


.82* 


.70* 


— 



Note . I = Learning Trial I: II *= Learning Trial II. 
* p < .05, two-tailed. 



activities for improving reading comprehension performance. 
With regard to the role of note-taking, subjects who took 
notes did not differ in comprehension, after a first learning 
trial, from subjects who only read the text. Moreover, subjects 
who simply reread the text performed as well on the second 
learning trial as subjects who reprocessed their notes, either 
by revision or rereading. The latter result extends earlier 
evidence showing repetitive reading to be an efficient reproc- 
essing strategy (e.g., Amlund et al., 1986; Barnett & Seefeldt, 
1989). 

As was expected from a verbal efficiency view of reading, 
as well as from a schema-based view of reading, above-average 
readers showed higher comprehension accuracy scores than 



Table 7 

Summary of the Stepwise Prediction of Sentence 
Verification Times for Explicit, Text-Implicit, 
and Script-Implicit Questions 



Question type/ 
variable 
entered 


R 


R 7 


Change 
in R 7 


Change 
in F 


Explicit 








12.43 


1. Decoding 
speed 


.461 


.213 


.213* 


2. Probe discourse 
accuracy 


.461 


.213 


.000 


0.02 


3. Domain-specific 
prior knowledge 
Text-implicit 


.485 


.236 


.023 


1.30 


1. Decoding 
speed 


.420 


.177. 


.177* 


9.88 


2. Probe discourse 
accuracy 


.438 


.192 


.015 


0.82 


3. Domain-specific 
prior knowledge 
Script-implicit 


.440 


.194 


.002 


0.11 


1. Decoding 
speed 


.419 


.175 


.175* 


9.78 


2. Probe discourse 
accuracy 


.422 


.178 


.003 


0.16 


3. Domain-specific 
prior knowledge 


.426 


.181 


.003 


0.15 



*/?<. 001 . 



did average readers after both learning trials, and prior knowl- 
edge was positively related to reading comprehension. Com- 
bining two different theoretical assumptions, this result dem- 
onstrates large roles for basic reading processes and specific 
knowledge in reading comprehension that are relatively in- 
dependent of the specific type of review activity. 

Several results in this study indicate more specific effects of 
text reprocessing on reading comprehension. For example, 
reprocessing facilitated comprehension of information that 
was both explicitly stated in the text and inferable from the 
text but did not facilitate comprehension for information that 
had to be inferred from a reader’s prior knowledge base. This 
result suggests that the activity of reprocessing was used 
primarily to learn more about factual knowledge and the ways 
in which it can be interrelated, whereas students were less 
engaged in elaborating their already existing schemata during 
reprocessing. In this regard, it is interesting that verification 
times were longer for script-implicit sentences than for text- 
implicit sentences and that sentences that covered factual 
information explicitly stated in the text were verified even 
faster. This result suggests that implicit information requires 
more inferencing than explicit information *abd that when 
information has to be activated from the prior knowledge 
base, additional effort is indicated by increasing processing 
times. 

An interaction of question type with reading ability indi- 
cated a further specification of reprocessing effects. Above- 
average readers showed relatively greater improvement from 
the first to the second learning trial on script-implicit multiple- 
choice questions. In contrast, average readers showed rela- 
tively greater improvement on text-implicit questions. This 
result indicates that text reprocessing helped average readers 
to learn more information that had to be inferred from the 
text, whereas above-average readers took the reprocessing trial 
as an opportunity to combine text information with their 
prior knowledge base. 

When the dependent variable was more time-sensitive, as 
was the case for the sentence- verification task, average readers 
comprehended at a lower level, regardless of information type. 
It is interesting that the sentence- verification accuracy data 
showed a less homogeneous pattern for average readers, in- 
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dicating that average readers who reread their notes answered 
as many sentences correctly as above-average readers when 
the sentences covered explicit or text-implicit information. 
One might speculate that rereading one’s own notes helps 
less-skilled readers to improve comprehension, whereas 
skilled readers seem to be less affected by the type of reproc- 
essing strategy applied. This hypothesis remains to be tested, 
and one way to do this might be to choose longer retention 
intervals and comprehension tests that provide fewer retrieval 
cues. 

The availability of retrieval cues might also be the reason 
for the weak relation between the number of noted idea units 
and comprehension accuracy after a first learning trial (r = 
0.16), as well as after reprocessing (r = 0.24). Studies that 
have reported stronger relations between the extensiveness of 
notes and comprehension of lecture information (e.g., Fisher 
& Harris, 1973; Kiewra & Fletcher, 1984) typically have used 
learning measures such as open-ended questions and free 
recall. In the present study, the cues provided by multiple- 
choice questions and sentence verifications were more spe- 
cific. 

Investigating the relative effects of reading components on 
reprocessing of text, we found probe-discourse accuracy to 
contribute significantly to comprehension. Knowledge also 
made a unique contribution to comprehension for explicit 
and script-implicit information and did not play a significant 
role when information had to be combined across the text. 
This result suggests that inference processes that are primarily 
text driven may be qualitatively different from inference 
processes that are knowledge driven. As others have suggested, 
knowledge-driven inferences are based on an automatic 
spreading activation process and are relatively effortless 
(Kintsch, 1988; Yekovich et al., 1990). In contrast, text- 
implicit inferences require the reader to maintain and com- 
bine several propositions in working memory, which is a 
relatively effortful activity and does not rely on prior knowl- 
edge, The ability to maintain discourse segments in working 
memory also seems to be a prerequisite to comprehending 
explicit and script-implicit information, and knowledge ap- 
pears to support these comprehension processes. 

In contrast to comprehension accuracy, working memory 
and prior knowledge did not contribute to sentence-verifica- 
tion times, whereas decoding speed was a significant predictor. 
The data indicate that decoding ability is related to compre- 
hension speed measures that seem to be independent of 
accuracy measures, 

In conclusion, rewriting notes, rereading notes, and reread- 
ing text were shown to be equally effective strategies for 
improving comprehension of explicit and text-implicit infor- 
mation, and individual differences in basic reading processes 
and prior knowledge contributed more to reading comprehen- 
sion than did the specific reprocessing activity. The results 
further suggest that proficient readers seem to learn more 
factual information and relate it to their schemata during 
reprocessing, whereas reprocessing might help less proficient 
readers compensate for their lower performance when infor- 
mation has to be inferred from the text. 

As cognitive differences among readers, such as working 
memory processing strategies and prior knowledge, are more 



thoroughly examined in text reprocessing, instructional tech- 
niques may be specified that improve comprehension relative 
to learner capabilities and relative to the type of information 
requested. 
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Appendix 

Analyses of the Decoding and Working-Memory Measures 



Word and Pseudoword Vocalization Task 

A word-type effect, FT 2,94) *= 98.30, p < 0.001, revealed longer 
vocalization latencies for low-frequency (662 msec; SD = 126) com- 
pared to pseudo words (837 msec; SD = 207). r(94) = 7.6, p < 0.001, 
but the latencies were not significantly longer for low-frequency than 
for high-frequency words (626 msec: SD = 97), /(94) = 1.56. p > 
0.05. 

A word-length effect, FT3, 141) = 53.31, p < 0.001. indicated 
shorter vocalization latencies for two-syllable-five-letter (659 msec; 
SD « 106) than for two-syllable-eight-letter words (745 msec; SD «= 
160), /( 1 4 1 ) = 7. l,p< 0.001, whereas three-syllable-cight-letter words 
were not significantly vocalized slower (754 msec; SD = 176) com- 
pared to two-syllable-eight-letter words, |/| < 1. Moreover, two- 
syllable-five-letter words were not vocalized slower (668; SD = 109) 
than one-syliable-five-letter words, |/| < 1. 

Length interacted with word type, FT6, 282) ** 31.42, p < 0.001, 
indicating that the difference between two-syllable-five-letterand two- 
syllable-eight-letter words was more pronounced for pseudo words 
(181 msec; SD = 161) compared to both high-frequency (33 msec; 
SD = 47), r(94) = 1 1.28, p < 0.001, and low-frequency words (49 
msec; SD = 54), f(94) = 10.08, p < 0.001 . These vocalization latency 
differences were not significantly greater for low-frequency than for 
high-frequency words, r(94) = 1.22, p > .10. The mean vocalization 
latencies for each word type and length category are displayed in 
Table Al. 

Table A 1 

Mean Vocal izat ion Latencies and Standard Deviations 
in Milliseconds for Words and Pseudowords 
Varying in length 











Word 


length 










5-1 


5 


-2 


8-; 


2 


8- 


3 


Frequency 


M 


SD 


M 


SD 


M 


SD 


M 


SD 


High 


606 


82 


606 


92 


640 


107 


656 


124 


Low 


644 


108 


632 


108 


681 


138 


689 


16! 


Pseudoword.. 


758 


155 


745 


143 


926 


283 


928 


284 



Mote. The first digit in word length categories indicates the number 
of letters, and the second digit indicates the number of syllables. Only 
correct responses ( M = 97.5% correct, SD = 1 .8%) are included. 



Table A2 

Mean Percentages and Standard De\’iations of Correctly 
Recalled Targets for Probes Vary ing in Distance and 
Segment Type 



Segment type 

Current Preceding 



Probe distance 


m 


SD 


M 


SD 


Near 


87% 


13% 


64% 


17% 


Far 


38% 


17% 


36% 


15% 



Similar to data reported by Haenggi & Perfetti (1991). a word- 
length effect in decoding was indicated by a letter effect and vocali- 
zation time increased most from five- to eight-letter pscudowords. 

Probe Discourse Task 

Target words were better recalled for nearf74.5%; SD ~ 1 2.3) than 
for far probes (37. 1 %; SD = 14.5). FT 1.47) = 333.9, p < 0.001, and 
targets were recalled at a higher rate when the probe occurred in a 
current text segment (62.6%; SD = 1 1 .3) compared to probes from a 
preceding segment (50.1%; SD = 1 3.5), FT 1 .47) = 65.74, p < 0.001. 

A probe-distance by segment-type interaction, FT 1,47) = 34.66, p 
< 0.001, indicated that near targets were better recalled following a 
probe word in a current compared to a preceding text segment, r(47) 
** 13.66, p < 0.001, whereas recall accuracy did not significantly 
differ between the current and preceding segment types when probe 
distance was far, /(47) = 1, 1 8, p > 0. 1 . The mean recall accuracy data 
for each probe distance and segment type are summarized in Table 
A2. 

The probe memory data suggest that subjects could use the sen- 
tence boundary as a cue for recoding only when the discourse segment 
to be held in working memory was short enough. This result is 
consistent with earlier work by Goldman, Hogaboam, Bell, and 
Perfetti (1980). 
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Abstract . This study was primarily concerned with the literary 
meaning-making experience. In all, 216 protocols were collected from 
36 students (18 7th graders and 18 11th graders). Over several weeks, 
each student engaged in a series of think-aloud sessions during which 
each read two short stories, two poems, one science selection, and one 
social studies selection. All think-alouds were tape recorded and tran- 
scribed to permit qualitative analyses of students* language and 
thought as evidence of the process of interpretation. 

Findings identify four recursive stances readers take in relation to 
the text, with each adding a somewhat different dimension to the un- 
derstanding of the entire piece. They are: being out and stepping into 
an envisionment, being in and moving through an envisionment, step- 
ping back and rethinking what one knows, and stepping out and ob- 
jectifying the experience. In addition to using these stances, students* 
overall orientation toward literary meaning-building involved explor- 
ing a horizon of possibilities, while the reading of other material in- 
volved maintaining a point of reference. It was this distinction which 
seemed to underlie the different approaches to understanding the stu- 
dents called upon when reading for literary and informative purposes. 



The decade of the 1980s has been marked by calls for increased attention 
to students’ thinking and reasoning about what they are reading in all of 
their subjects, across the curriculum (Applebee, Langer, & Mullis, 1987, 
1989, 1990; Boyer, 1983; Commission on Excellence in Education, 1983; 
National Governors’ Conference, 1990). For English teachers, this has 
meant a focus on the reading of literature — calling it back to the center of 
concern after several decades of benign neglect. 

Yet there is considerable disagreement about the role that literature 
should play in the K— 12 curriculum. For many, the teaching of literature 
is often treated only as a way to introduce students to the cultural knowl- 
edge, the great thoughts, and the high culture of our society (Bloom, 
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1987; Cheyney, 1987; Hirsch, 1987; Ravitch &c Finn, 1987); its role in the 
development of the sharp and critical mind is often ignored. However, 
there is evidence from a number of sources that narrative thought is part 
of the well-developed intellect, and that it is important in contexts other 
than literature. 

Putnam (1978) suggests that literary understanding, with its attention 
to how people live, is an important part of scientific thought, and that the 
imagination and sensibility that accompany literature are essential ele- 
ments in scientific reasoning. Similarly, Dworkin (1983), in an article on 
law and literature, argues that the understanding of law can be enhanced 
by literary approaches to interpretation.- Elstein, Shulman, and Sprafka 
(1978), in a study of ways in which doctors arrive at diagnoses, have 
shown that doctors who usually use “logic” to reach a diagnosis turn to 
“storytelling” in solving difficult problems; narrative thought is a produc- 
tive alternative for them. Orr (1987a, 1987b) studied the ways in which 
technicians who repair highly complex machines go about building their 
understanding of the problem. He similarly reports that stories are re- 
membered and told during the diagnosis of the problem, and that notions 
of the repairers’ expertise grow out of their ability to engage in and ab- 
stract information from narratives. Generalizing across fields of inquiry, 
Bruner (1986) argues that there are two modes of cognition — narrative 
and paradigmatic— each with its own way of viewing reality. Full under- 
standing, he suggests, is better achieved by using both the ordered 
thought of the scientist and the humanely inquisitive thought of the story- 
teller. The paradigmatic mode offers facts, objectivity, logical proofs, and 
reasoned hypotheses, while from the study of literature we come to un- 
derstand the “vicissitudes of human intention” (p. 17). Britton (1983) sim- 
ilarly contrasts the rule-governed thought of the scientist with the many- 
sidedness of literary thought, suggesting the complexity of the latter is 
necessary for understanding the human experience. It is the focus on the 
human situation described by both Bruner and Britton that draws the in- 
dividual into the act of thinking — as she or he experiences the events, 
emotions, and intricacies of human life. The experience, in turn, becomes 
available for analysis and reflection. 

Such issues underlying narrative understanding are critical for the next 
generation of work in the teaching of literature, yet they are not suffi- 
ciently well developed to drive new conceptualizations of the role of liter- 
ature in the curriculum, nor of how to teach it. We need to learn more 
not only about distinctions between literary and other readings, but also 
about the nature of the meaning-making process itself. 

Klemenz-Belgart (1981) suggests the need to use current theories of 
text understanding in studying ways in which readers comprehend text, 
while theorists as diverse as Bahktin (1973), Culler (1980), and Barthes 
(1974) call for increased understanding of the conventions readers refer 
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to during the sense-making activity. Culler (1980), for example, suggests 
the need for a reorientation of focus from corpus to interpretive strat- 
egies: 

To account for the form and meaning of literary works is to make ex- 
plicit the special conventions and procedures of interpretation that en- 
able readers to move from the linguistic meaning of sentences to the 
literary meaning of works. ... In brief, 1 am arguing that if the study 
of literature is a discipline, it must become a poetics: a study of the 
conditions of meaning and thus a study of reading, (p. 49) 

It was to better understand the act of literary reading, particularly in 
students, that this study was undertaken — to begin to describe the nature 
of literary understanding and to relate it to the kinds of meaning-building 
students engage in when they read other kinds of text. Rather than exam- 
ining the expressed content of students’ understandings, the focus was on 
the approaches they use in developing that understanding during the 
reading both of literary and of informative works. The study asked this 
question: How do students go about understanding literature, and how is 
this related to the ways they make meaning when reading science and so- 
cial studies selections? 



Theoretical Starting Points 

When one begins a study of this sort, there are a number of theoretical 
assumptions that are implicit in the conceptualization of the work. In this 
study, there are at least three such assumptions: (1) that reading is an ex- 
perience of growing understandings that change over time, and thus is es- 
sentially an interpretive act (I mean this in the sense used by reading 
theorists, and not in the sense of formal interpretation discussed by liter- 
ary critics); (2) that although meaning resides in the reader, not the text, 
readers follow certain conventions which are signaled by linguistic fea- 
tures in the piece being read; and (3) that approaches to reading are 
functionally driven, informed by pragmatic needs to personalize or objec- 
tify experience, and affect the universe of discourse the reader selects to 
guide envisionment-building. 

Reading as an Act of Envisionment Building 

Reading is sense-making, an act of becoming — where new questions, in- 
sights, and understandings develop as the reading progresses, while un- 
derstandings that were once held are subject to modification, 
reinterpretation, and even dismissal (Anderson, Spiro, 8c Montague, 
1977; Goodman, 1970; Iser, 1978; Rumelhart, 1975; Spiro, Bruce, 8c 
Brewer, 1980; Suleiman, 1980). At any point in a reading, the individual 
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has a local envisionment, a personal text-world embodying all she or he 
understands, assumes, or imagines up to that point in the reading (see 
Fillmore, 1981; Kay, 1987; J. Langer, 1985, 1986, 1987). However, this is 
momentary — subject to change in response to subsequent thoughts that 
may be (but are not necessarily) text-based. The final envisionment, then, 
is never the sum of previous traces, but is instead an evolving whole, 
which itself is subject to change well after the pages have been removed 
from sight. In this view, reading is interpretation (Sontag, 1956), and if 
one wishes to understand this act of interpretation it is necessary to exam- 
ine the reading experience across time, as the reader traverses the course 
of meaning-making. 

Relationships Between the Reader and the Text 

The question posed by this study, about the different kinds of thinking 
readers experience in reading literary and other kinds of texts, suggests 
from the outset the belief that the text cannot be ignored in its contribu- 
tion to the reader’s meaning; although the reader’s purpose plays a major 
role in how the meaning is orchestrated, the text itself influences the 
reading. Texts function semiotically, providing an array of signs and 
guideposts to evoke ideas and images which are helpful in signaling and 
underscoring particular ideas that are important to the reader’s construal 
of a particular piece — texts follow conventions that are helpful in arous- 
ing readers’ thoughts in ways that go beyond what words alone can do 
(Grice, 1975; Iser, 1974; Searle, 1969). As Pratt (1976) and Smith (1978) 
have argued, the issue of literary and non-literary readings requires ex- 
amination of the pragmatic function of the texts themselves. A social rela- 
tionship develops between reader and text (Booth, 1988; Hunt & Vipond, 
1985; Vipond &: Hunt, 1984; Vipond, Hunt, & Wheeler, 1987) in which 
readers, as in everyday discourse, establish a certain social orientation rel- 
ative to their perceptions of the text. While readers may respond differ- 
endy to texts depending upon their perceptions of the implied text or im- 
plied audience (Booth, 1988; Rabinowitz, 1987), they make the choice of 
how to orient themselves, and this determines whether their approaches 
toward understanding a particular text will proceed in a predominandy 
literary or non-literary manner. When readers treat the text in a literary 
manner, as S. Langer (1942) suggests in her work on the symbolic func- 
tion of language, from the very first few words they are drawn into the 
text, leaving the everyday world behind. They seek to identify the genre 
from the moment reading begins, and these early hypotheses, although 
subject to change, help shape how they read and the meanings they create 
(J. Langer, 1986). Thus, the reader “weaves a web of feelings, sensations, 
images, and ideas between the reader and the text” (Rosenblatt, 1978), 
and a study of the growth of understanding must explain, not ignore, that 
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relationship. To examine this, the present study used a mix of literary 
and informative selections, but allowed the readers to define their own 
orientations in relation to each text. 

Objective and Subjective Realities 

S. Langer (1967) describes human feelings and understandings as ema- 
nating both from outside the individual and from within. She refers to 
these as objective and subjective experience, explaining that both are part 
of the process of making meaning. However, the different starting points 
(the objective outer world and the subjective inner world) lead to related 
but somewhat different symbol systems and approaches toward meaning. 

Many scholars who have examined language and meaning have distin- 
guished between these two universes of discourse and their accompanying 
techniques for representing experience. S. Langer (1942) herself, in her 
work on the process of symbolization, distinguishes between discursive 
and presentational techniques; Rosenblatt (1978), focusing on the reader’s 
role, distinguishes between efferent and aesthetic approaches to reading; 
and Britton (1970), in his work on the development of writing abilities, 
distinguishes between participant and spectator roles with their associated 
transactional and poetic techniques. Although developed for different 
purposes, each set of concepts deals in some way with qualitative differ- 
ences between literary and non-literary experiences. Each describes on 
the one hand a situation where the person holds meaning apart, in quest 
of a more analytic understanding, and on the other hand a situation 
where the person becomes personally enmeshed in the text world, re- 
sponding on a more subjective plane. The first can be considered the way 
of scientific reading (as in Bruner’s notion of paradigmatic thought), 
while the other can be considered the way of literary reading (as in 
Bruner’s narrative thought). The goal of this study was to learn more 
about the ways in which these modes of understanding occur when stu- 
dents read for literary or informative purposes. 



Related Studies 

Although there have been many specific studies of “response” to liter- 
ature, these have focused primarily on content analyses of expressed re- 
sponses rather than examining the knowledge and strategies that 
contribute to students’ understandings. From these studies we know: that 
younger children focus on “action” rather than “interpretation” (Ap- 
plebee,. 1978; Mason, 1974; National Assessment of Educational Progress, 
1973); that ability and achievement differences have little effect on the 
content of expressed responses except as by-products of comprehension 
problems (Angelotti, 1972; Auerbach, 1974; Faggiani, 1971); that older 
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students are more likely to give “interpretive” or “formal analytic” re- 
sponses (Cooper, 1969; McGreal, 1976; Purves, 1973); that girls are more 
likely than boys to verbalize “engagement” or “involvement” in a story 
(Purves, 1973); that the content of students’ responses varies over time 
during the course of reading or rereading (Angelotti, 1972; Beach, 1972; 
Britton, 1954; Squire, 1964); and that there are strong effects tied to the 
particular literary selection chosen as stimulus (Angelotti, 1972; Cooper, 
1969; Comaby, 1974; Purves, 1973; Weber, 1973). 

A small number of studies have focused on the strategies readers use 
when approaching text. Hunt and Vipond (1985) distinguish among 
story-driven, information-driven, and point-driven orientations, indicat- 
ing ways in which the different purposes for reading affect response, 
while Vipond and Hunt (1984) illustrate particular cognitive strategies as- 
sociated with point-driven readings of literary texts. Jacobsen (1982) de- 
scribes college students’ unwillingness to suspend disbelief or apply their 
own experiences in order to enter the text-world or “potential space” of 
short stories, and Dillon (1982) describes three styles of reading based on 
readers’ perceptions of the chronological sequence or event chain of the 
story, arguing that readers understand life and literature in similar ways. 
Rather than focusing on the expressed content, as much of previous work 
has done, there is compelling theory concerning language and thought 
that supports further study into the ways students create their under- 
standings. To develop this theory, this study focused on the act of reading 
from the reader’s vantage point. 



The Study 

In particular, this qualitative study sought to describe the ways in which 
middle and high school students create meanings when they are reading 
for literary or informative purposes. The student-informants attended 
two cooperating school districts, one an inner city and one a suburban dis- 
trict. The superintendents, principals, and English department chairper- 
sons all expressed an interest in supporting Literature Center project ac- 
tivities, of which this study was a part. One middle school and one high 
school were selected in each district. The suburban schools were in 
middle-class bedroom communities; students were generally bused to 
schools; approximately 49 percent of the high school graduates went on 
to 4 year college; and 29 percent went on to other forms of post-secon- 
dary school education. The city schools had all of the characteristics usu- 
ally associated with inner-city environments. Although the community was 
integrated, the children were predominantly minority and predominandy 
from poor families; businesses and residences were nearby; children gen- 
erally walked to school; only 27 percent of the students went on to four 
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year college; and 39 percent went on to other forms of post-secondary ed- 
ucation. 

To enlist participants, the teachers and research assistants described 
the project to the students, inviting them to become involved; all 7th and 
11th graders and their parents also received letters requesting their con- 
sent for participation. Of those who responded, 18 were chosen at each 
grade level, with 9 students at each grade in each school (3 considered as 
average, 3 as above average, and 3 as below the norm for their grade in 
their school based on statewide standardized test results corroborated by 
teacher judgment). Thus, the 36 students were selected to permit us to 
learn about literary meaning-making strategies across a cross-section of 
students from two diverse districts. 

Materials 

A review of available short stories, poems, science selections and social 
studies selections led to consideration of some 80 works, all of which were 
typical of those found in school collections and magazines designed for 
students in junior and senior high school, and each of which could be 
read by both 7th and 1 1th graders. The initial selection was narrowed to 
8 poems, 8 stories, 4 social studies texts, and 4 science texts which were 
then field-tested for appropriateness. The field tests consisted of students 
reading and discussing the texts, and indicating whether they thought the 
texts would be familiar to and of interest to other students like them- 
selves. The chosen texts did not present extreme difficulty for any of the 
field-test students to read, and met their criteria for recommendation. 
The final selections were: “Man by the Fountain” (Hebbelinck, 1973: 
short story), “I See You Never” (Bradbury, 1973: short story), “The Fish” 
(Bishop, 1968: poem), “Forgive My Guilt” (Coffin, 1966: poem), “Birth of 
the Moon” (Birnbaum, 1986: science) and “E. R. A.: Triumph of the 
American Nation” (Todd Sc Curd, 1986: social studies). 

Procedures 

Over a series of meetings, each student completed a training session and 
think-aloud protocols on six texts. The think-aloud procedure clearly 
does not capture all of a reader’s thoughts and strategies and creates an 
artificial reading situation (limitations are well-described by Ericsson 8c 
Simon, 1984; and Afflerbach & Johnston, 1984), but it has proven to be 
an effective technique for examining how students orchestrate their read- 
ing and writing strategies over time (Flower Sc Hayes, 1980a; Hayes 8c 
Flower, 1980b; Hunt Sc Vipond, 1985). In an earlier study (Langer, 
1986), I compared think alouds and stimulated recalls during reading and 
writing, and found that think alouds tended to be longer and contained 
some of the more momentary decisions that were no longer remembered 
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during stimulated recalls. The earlier study, also found that the think- 
aloud procedure did not lead to distortions in students’ comprehension of 
what they had read or in the texts that they wrote. 

During the first meeting, each student was introduced to the think- 
aloud procedure, and practiced it with preselected short stories, poems, 
social studies texts, and science texts until he or she felt comfortable w'ith 
the experience. The students were encouraged to verbalize their thoughts 
when they felt comfortable doing so, rather than at a predetermined 
boundary such as at the end of a sentence or paragraph. This was done to 
minimize the distractive effect of the think-aloud procedure on the devel- 
opment of meaning. 

Each student participated in 6 think-alouds, in response to 2 short sto- 
ries, 2 poems, 1 science text, and 1 social studies text. Order of texts was 
counterbalanced. Each session was approximately 40 minutes in duration. 
Students were asked to read each selection in the manner in which they 
generally read works of that sort. Rather than prompting the students to 
read the poems and short stories for literary purposes (for the experi- 
ence) and the science and social studies selections for informational pur- 
poses, the choice of orientation was left to the students themselves; they 
made the choice based on the influence of the text’s structure and the 
purpose they set for themselves. As will become clear in discussing the re- 
sults, the literary and informative texts did evoke consistently different 
orientations, reflecting different purposes for reading. 

Analyses 

All analyses were qualitative in nature, involving successive steps of data 
reduction and verification. To accomplish this, each of the transcripts was 
carefully read, first separately and later in comparison with the other 
transcripts, in a search for patterns of “on-line” concerns the students 
voiced during reading. Once identified, specific evidence for these pat- 
terns was sought by returning to the transcripts for examples in the stu- 
dents’ own language. A recursive process of refining patterns and return- 
ing to the transcripts for evidence was repeated until the patterns 
identified accounted for all of the individual comments in the student 
transcripts. The endpoint of the recursive analyses was a final list of cate- 
gories along with students’ language samples, with each language example 
identified as to the time it occurred during the reading (beginning, mid- 
dle, or end). Annotation of data, recursive analyses, and identification of 
recurring themes and patterns provided qualitative reliability (Goetz & 
LaCompte, 1984). 

The transcripts from all of the readings of all the selections were ana- 
lyzed, and the findings reported are based on the complete set. Through- 
out this report, however, two pieces, a short story (“I See You Never”) 
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and a science selection (“Birth of the Moon”), will be used as examples. 
They were selected because they provided the clearest and most con- 
sistent examples of the patterns represented across all the readings. 

“I See You Never" (Bradbury, 1973) is about Mr. Ramirez’s imminent 
deportation to Mexico. He is an illegal alien who has clearly enjoyed living 
and working in the United States and would like to stay. However, he 
over-extended the tenure of his visa, was apprehended by the police, and 
was escorted to his apartment in order to pick up his belongings. Mrs. 
O’Brian, his landlady, seems deeply moved by his predicament, slowly re- 
alizing, as Mr. Ramirez had pointed out, that they would never see each 
other again. 

“Birth of the Moon” (Birnbaum, 1986) is about the impactor theory of 
the moon’s formation, positing that billions of years ago a planet-like ob- 
ject with a core of iron impacted with the earth, sending hot gasses and 
other material into space. These materials held together, forming the 
moon. The theory explains that the moon and earth have both similar 
and different chemistries because of the ways in which particular chem- 
icals were deposited or interacted as a result of the original impact. 



Results 

Findings in this study concern the meaning-making process itself and the 
approaches the students used when reading for literary or informational 
purposes. These findings indicate that the process of reading is one that 
involves a variety of changing stances that the reader takes toward the 
text. The focus of the reader’s concerns in each stance differs consider- 
ably. The sections that follow will First explore the nature of these stances, 
and then turn to the characteristics that differentiated readers’ ap- 
proaches to reading for literary as compared to informative purposes. 
The focus throughout will be on patterns that held across the diverse 
group of readers, in spite of their differences in age, levels of achieve- 
ment, and types of schools and communities. 

Stances in the Process of Understanding 

Analyses of the think-aloud reports focused on the ways in which readers 
engaged in creating meanings as they read. These analyses indicated that 
as the readers developed their meanings across time, their stances (the 
ways in which they related to the text) changed, with each stance adding a 
somewhat different dimension to the readers’ understanding of the entire 
piece. These stances were recursive rather than linear (they had the po- 
tential to recur at any point in the reading) and were a function of vary- 
ing reader/text relationships. 
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The four major stances used in the process of understanding as they 
emerged in this study were: 

* Being Out and Stepping Into an Envisionment — In this stance, readers 
attempted to make contacts with the world of the text by using prior 
knowledge, experiences, and surface features of the text to identify 
essential elements (e.g., genre, content, structure, language) in order 
to begin to construct an envisionment. 

* Being In and Moving Through an Envisionment — In this stance, readers 
were immersed in their understandings, using their previously- 
constructed envisionment, prior knowledge, and the text itself to 
further their creation of meaning. As they read more, meaning- 
making moved along with the text; readers were caught up in the 
narrative of a story or were carried along by the argument of an in- 
formative text. 

Stepping Back and Rethinking What One Knows — In this stance, readers 
used their envisionments of the text to reflect on their own previous 
knowledge or understandings. Rather than prior knowledge inform- 
ing their envisionments as in the other stances, in this case readers 
used their envisionments of the text to rethink their prior knowl- 
edge. 

* Stepping Out and Objectifying the Experience — In this stance, readers 
distanced themselves from their envisionments, reflecting on and 
reacting to the content, to the text, or to the reading experience it- 
self. 

The nature of these stances and how they contributed to readers’ under- 
standings will be discussed below, with examples from the verbal reports 
of various students. The examples have been selected in order to provide 
a range of illustrations of readers’ concerns within each stance. A more 
extended, coded selection from a think-aloud protocol of “I See You 
Never” is included in the appendix as an illustration of the ways in which 
the stances interplay during a single reading. 

Being Out and Stepping Into An Envisionment 

Being out and stepping into an envisionment describes a somewhat dis- 
tant relationship between the reader and the text. In this stance, the read- 
er tries to establish a context for understanding the piece, usually by 
asking questions and making associations. This stance occurred as the 
reading began, and again when a reader encountered unexpected or un- 
familiar vocabulary or information at any point in the reading. It was a 
time when readers envisionments did not cohere, because they had not 
developed a sufficient core of text-related knowledge to build upon. In 
literature, readers tried to make contact by examining such features as 
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characters, plot, and setting — and how they interrelated; in dealing with 
the informative selections, readers tried to make contact by figuring out 
what the point of the piece would be or what the topic was about. In each 
case, they sought information from the text to put together with their 
background knowledge to get enough information to “step in." 

Examples from literary texts. When the readers encountered stories, one of 
the things they tried to establish was who the characters were — they asked 
questions and made hypotheses about the characters’ identities, where 
they came from, how they looked, and what they were like. For example, 
Susan simply noted the appearance of characters and made an initial hy- 
pothesis about Mr. Ramirez’s ethnicity when she began to read “I See You 
Never.” (In the following transcript segments, the actual text is un- 
marked, and the students’ comments are italicized.) 

The soft knock came at the kitchen door, and when Mrs. O’Brian 
opened it okay, there's one character, Mrs. O’Brian there on the back 
porch were her best tenant, Mr. Ramirez, I don’t know, that sounds like a 
Spanish name and two police officers. Okay, so right there there’s four char- 
acters — one policeman on each side of him. 

Jack’s attempt to step into an envisionment went beyond identification, to- 
ward an understanding (however superficial) of the character: 

From the first, he had made big money. He saved some of it, and he 
got drunk only once a week, a privilege that to Mrs. O’Brian’s way of 
thinking every good workingman deserved, unquestioned and unre- 
primanded. Okay, so she’s not really, she’s not a conservative woman per se. 

She thinks it’s ok for him to save some money and get drunk on the rest if that’s 
what he wants to do. 

When in this stance, readers sometimes also noted the genre, the form, 
or the style of the piece — any feature of the text that might help them re- 
late the text to what they already knew. For example, when Marguerite 
began to read she was surprised by the language of the title and tried to 
understand it before moving on: 

I See You Never. That's a pretty strange title. It doesn’t, I mean gram- 
matically, it doesn’t make sense. I would say, “1 Never See You,’’ But, I See 
You never almost sounds like a foreigner saying this, or someone who doesn’t 
know English that well. 

Susan used a different approach to making contact, noting that the di- 
alogue in the story differed from the internal monologue used in other 
short stories she had read: 

Mr. Ramirez just stood there, walled in and small. “Why, Mr. 
Ramirez," said Mrs. O’Brian. This is going to be different from the other 
ones I’ve read. This one has conversations in it. 
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In being out and stepping into an envisionment, readers sometimes 
used hypotheses about the relationships among the characters as a way to 
establish an initial understanding of the situation. Susan’s attempt to un- 
derstand the relationship between Mrs. O’Brian and Mr. Ramirez ex- 
emplifies this: 

He had arrived at Mrs. O’Brian ’s rooming house more than two years 
earlier and had lived there ever since. So, I wonder if Mrs. O'Brian and 
Mr. Ramirez are really good friends. 

Other readers used details of time or setdng to relate the story to situa- 
tions that were familiar, giving themselves a way to begin to construct an 
envisionment. Focusing on the setting, Jack said: 

He had come by bus from Mexico City to San Diego. Okay, so this is tak- 
ing place in California. I think San Diego is in California. There he had 
found the clean little room with glossy blue linoleum and pictures and 
calendars on the flowered wall. Okay, so it’s a nice quaint California room- 
ing house. Flowered walls, that paints a pretty picture in my mind. 

Familiar situations used to contextualize the story were not always 
positive, as is evident in Robert’s comments: 

From the first, he made big money. He saved some of it, and he got 
drunk only once a week, I wouldn’t want to get drunk. And that’s often. I 
really don’t like drunks. 

Although this stance occurred most often early in a story, it also re- 
sulted later when students encountered vocabulary or situations un- 
familiar enough to take them out of the story to puzzle over the meaning 
of a word (e.g., I wonder what adobe is? and What does ‘ reprimanded ’ mean?). 

In each of these examples, the students used the meaning sources at 
their disposal, their own experiences and knowledge as well as the text, to 
gain enough understanding to step into the world of the story. 

Examples from informative texts. Being out and stepping into an envision- 
ment also characterized an important stance when students were reading 
for informative purposes. In this stance, students sought to identify the 
topic, the meaning of a particular segment, the genre and organization of 
the piece, or the surface language in order to have a way to step into the 
world of the text. 

From the moment they began to read, many students tried to make 
sense by focusing on what the topic might be and what they already knew 
about it. Marguerite’s comments when reading “Birth of the Moon” ex- 
emplify this: 

A planet the size of Mars comes hurtling through space at 25,000 
miles an hour and smashes into the earth. They're talking about the Big 
Bang theory or something. 



National Library of Education 



66 



70 



Reading Comprehension 



The Process of Understanding: Reading for Literary and Informative Purposes 

Lesley similarly began by trying to establish the topic: 

Rocks vaporize. And a jet of hot gas squirts violently into outer space. 

A bright, hot flash lights up our solar system. Maybe it’s explaining how 
the moon came about. Like the beginning. 

In this stance, readers were also aware of whether the language and vo- 
cabulary of the text clarified their understandings or whether it got in the 
way. Marguerite found the descriptions helpful: 

Earth is blasted out of shape. Rocks vaporize. These are really good de- 
scriptions. I can even see like bombs bursting in air. 

Cara, on the other hand, wondered: Well, what are neutrons ? 

Students also drew on their previous knowledge and experiences to 
help them develop initial understandings of the text. For example, when 
reading about manganese and silicon, Robert stretched to make a connec- 
tion from what he knew to the unfamiliar chemicals he was reading 
about: 

The rocks were called (also contained) manganese and silicon, just like 
earth. Silicon reminds me of silicone, and at home we have to keep putting sil- 
icone inside the windows because the water always gels through every room. 

To sum up this stance, when readers were trying to step into an envi- 
sionment, they .drew on both their background experiences and the text 
itself to help them enter into the world of the text. In “I See You Never,” 
they were most likely to focus on the people, place, and situation, while in 
“Birth of the Moon” they were more likely to focus on information that 
might identify the topic or establish a relationship between the text and 
what they already knew. In each case, they used both the information 
from the text and their background knowledge to gain enough ideas to 
“step in.” Although being out and stepping into an envisionment oc- 
curred primarily at the beginning of a reading, readers also entered this 
stance during the middle of reading, when characters, informauon, lan- 
guage, or ideas were introduced that were unfamiliar enough to take 
them out of their envisionments — requiring the period of acquaint- 
anceship afforded by this stance. And, if new or totally unexpected infor- 
mation were to occur at the end (which was not the case in any of the 
pieces we used), it is conceivable that being out and stepping in could 
occur at that point in the reading as well: 

Being In and Moving Through an Envisionment 

Being in and moving through an envisionment describes the engaged mo- 
ments when readers used their personal experiences and knowledge as 
well as the text to push their envisionments along — where meanings begot 
meanings. In this stance readers already had gained initial understand- 
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ings of particular aspects of the piece, and were using the ongoing text to 
build an evolving envisionment. In each domain, they continued to devel- 
op their understandings beyond the superficiality of the “being out and 
stepping in” stance, elaborating upon and making connections among 
their ideas as the text progressed. 

Examples from literary texts . In the reading of “I See You Never,” this stance 
was marked by readers* comments about their increasing understanding 
of the story, whether of the characters, the situation, or the actions taking 
place. We can see this in Robert’s comments about his understanding of 
Mrs. O’Brian: 

She remembered the slow dragging horses and the parched jack rab- 
bits on the road. She remembered the iron mountains and the dusty 
valleys and the ocean beaches that spread hundreds of miles with no 
sound but waves. I can see now , she 's feeling sorry for Mr. Ramirez y cause y 
like the conditions in Mexico aren't as good as they are in San Diego — no cars , 
no buildings , nothing . That would be terrible. 

Susan s comments similarly reflect her developing understanding of 
how Mrs. O’Brian and Mr. Ramirez feel about the situation: 

I’m sure sorry, Mr. Ramirez, she said. I guess she doesn't sound like she 
wants him to leave , and 1 don't think he wants to go either . 

In this stance, readers also commented on their own personal reactions to 
the situation as they see it, as in Tanya’s remarks: 

Mrs. O’Brian, I see you never. I see you never. The policemen smiled 
at this, but Mr. Ramirez did not notice it, and they stopped smiling 
very soon. Don't the policemen like have any feelings toward this man f He 
wants to stay so badly . And he makes all this money. 

Other comments in this stance reflected the ways in which readers were 
connecting past envisionments to their present reading, as Jack did in the 
following: 

She remembered the silent towns, the warm beer, the hot thick foods 
each day, the beer the beer reminds me of him getting drunk once a week , the 
hot thick food each day reminds me of the many courses he might have had y the 
silent towns, silence loneliness , kind of symbolizes Mr. Ramirez. Mr. Ramirez 
doesn t say too much. The only time when he really shows affection or anything 
is when he says thank you , unpacked , like 7 have my bag all ready , here's the 
key Mrs. O Brian t and still he calls her Mrs. 0'Brian y which means he has re- 
spect for her y I'm sure. 

Susan s comments similarly reflected how she was weaving the parts to- 
gether in this stance: 

He reached out his hands and took her hand fervently, shaking it, 
wringing it, holding to it. “Mrs. O’Brian, 1 see you never, 1 see you 
never. Thats where the title of the story came from. I don't think he speaks 
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English very well. I'm not sure though, but the way he just said that, I don’t 
think he can speak English very well. 

Jack, on the other hand, considered Mrs. O’Brian’s return to the table 
from her children’s perspectives as well as her own, illustrating his inter- 
pretation of the characters as well as his sharp judgment about them: 

I bet all her five kids feel bad because they want their dinner, don't they, and 
then it’s cold, they’re complaining, they’re brats . . . "Hurry up Mom, said 
one of her sons. It’ll be cold.” Oh, shut up. This kid's making me mad. It’ll 
be cold. I mean sometimes you gotta bear with it. I mean, come on, this guy is 
leaving forever. I'm sure she really doesn’t care if her steak is cold, personally. 

Thus, when readers were being in and moving through an envision- 
ment during the reading of a story, their comments reflected an immer- 
sion in the envisionments they were creating, as they developed and deep- 
ened their understandings of the characters, situations, motives, and 
emotions that underlie the story. 

Examples from informative texts. Similarly in reading informative texts, read- 
ers who were in and moving through an envisionment were involved in 
refining their understandings and making connections as they moved 
through the text. 

Thus Lesley linked what she had just read into her overall understand- 
ing of “Birth of the Moon”: 

A jet of hot gas squirted thousands of miles into space. So they’re telling 
us that when it hit the earth that parts of it clipped the moon, and that’s why 
they're alike and that's why they're different. 

As they clarified their understandings of the evolving text world, readers 
in this stance also commented on their interpretation of the logic of an 
extended presentation of information. The following segment of Susan’s 
transcript exemplifies this: 

The big crash blasted the impactor apart. Okay, so that means that when 
this object hit the earth, that this crash blew it apart so it was all in little pieces. 

Its iron core tore away from its rocky surface, Okay, it had iron: like 
metal stuff in the middle, and rock on the outside, but I thought the core was, 
oh, yeah, okay. That’s right. I thought the core was made of rack, but no, it's 
not. It’s made of iron. That's better now. and plunged straight into the cen- 
ter of Earth. Okay, so I wonder if that's how these, how we got that Stuff in 
.‘he middle, in the core, of our planet. Rocks vaporized. Parts of the earth's 
surface were ripped to bits too. Okay, so not only was the impactor 
blown apart, part of the earth was too. So, now I'm beginning to see. 

To elaborate their envisionments, readers in this stance also drew on 
related knowledge, as Marguerite did: 

And the moon rocks had very few volatiles (materials that boil away 
during the hot explosions, such as water, sodium, potassium, and 
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lead). Oh, maybe thats why there aren't life forms on the moon, because there's 
very little water, and you cant live without water . 

Jack similarly drew on related knowledge and experience, though of a 
more personal sort: 

How could they explain this confusing rock chemistry? Why were the 
moon rocks like earth, but different from it too? Well, not everything's 
the same . Even if it comes from, like a baby and a mother, I mean. I'm not ex- 
actly like my mom. I'm not anything like my mom, actually, Cause I'm a boy 
and she's a girl. But, I'm also different like everything else is different. It 
doesn't have to be exactly the same to come from it. 

To sum up this stance, in being in and moving through an envision- 
ment, readers drew on personal experiences and information in the text 
to move their understandings along. In “I See You Never,” they focused 
on their understanding of people and the relationships among them to fill 
out their understanding of the story, while in “Birth of the Moon” they 
tried to understand each new explanation by linking it to what they al- 
ready understood the text to have said. 

For the students and texts studied, the first two stances predominated, 
with the readers major attention focusing either on getting acquainted 
with the situ&tion or stepping in and moving through their envisionments. 
However, the next two stances also occurred consistently in the readings, 
even though less frequently. 

Stepping Back and Rethinking What One Knows 

Stepping back and rethinking what one knows describes the thoughts that 
occurred when readers who were already immersed in creating an envi- 
sionment stepped outside of that world for a moment and used their 
growing understandings as ways to reflect upon their own lives or their 
own knowledge. 

Examples from literary texts. Readers encounter many situations in which the 
world of the text causes them to rethink their prior knowledge and expe- 
riences. Marguerite, for example, adopts this stance when she encounters 
a description of food she has never tried: 

Mr. Ramirez saw the long table, laid with clean white linen, and set 
with a platter, cool shining glasses, a water pitcher with ice cubes float- 
ing inside it, and a bowl of fresh potato salad, and one of bananas and 
oranges, cubed and sugared. I don't think I've ever had bananas and 
oranges cubed. And I don t think I would like sugar on them, but maybe I 
would. It might be interesting to try. 

Tanya, on the other hand, stepped out of the text to reflect on the op- 
tions for dealing with strong emotions: 

She pulled the chair out and sat down. She picked up the shining 
knife and fork and started once more upon her steak. It never happened 
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to me y but I know I would feci like Mrs. O' Brian and not be able to eat my 
steak . She makes me see you don't have to pretend when you're feeling so sad. 

Examples from informative texts . Readers’ comments reflected a similar set of 
relationships to the text when they read for informative purposes. In this 
stance, they used their understandings of the text to inform (and often re- 
think and revise) the knowledge they already had. We can see this in 
Robert’s comments, in which he used information about the moon to re- 
think what he knew about Jupiter and Saturn: 

As a result a lot of material went into orbit around the earth. It 
formed a disk, sort of like a pizza. That reminds me of Jupiter or Saturn 
because they have rings of dust and stuff. I wonder if it isn't dust. Maybe it's 
like chemicals like in the moon , and maybe they are part of another impact or 
crash . 

Marguerite used what she read to rethink the little she had heard about 
moon rocks: 

In the 1960s and 1970s, astronauts on the Apollo missions brought 
back moon rocks to Earth’s laboratories for the very first time. I re - 
member hearing about that. I thought they were in a museum , but maybe they 
can't be. I wonder if they have radiation and volatiles that make them dan- 
gerous , or maybe they're really more like the earth than people think. 

And John used what he read about the moon to form an hypothesis about 
Haley’s comet: 

Our round, gray moon was formed. So, maybe this is how Haley's comet 
was formed too. 

In other stances, readers used their background knowledge and expe- 
riences to help them understand the text, but in this stance they used 
their envisionments of the text world in reverse — to help them reflect on 
something they knew, did, or felt before having read the text. In this 
sense, “being out and stepping into an envisionment” and “being in and 
moving through an envisionment” involved use of prior knowledge to in- 
form and enrich their envisionments, while in “stepping back and re- 
thinking what one knows” readers used their envisionments to reflect on 
and sometimes enrich their real world. There was a symbiotic relationship 
as readers’ emphases moved between “real world” knowledge and experi- 
ences and “text envisionments,” permitting each to illuminate the other. 
Thus, the degree of elaboration of readers’ responses influenced the 
richness of both text and personal knowledge. 

Stepping Out and Objectifying the Experience 

In the fourth stance, stepping out and objectifying the experience, read- 
ers distanced themselves from the envisionments they had developed. 
They used this stance to reflect on the reading activity, their understand- 
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ings, and their reactions. They also commented on the text and the read- 
ing experience itself. 

Examples from literary texts. Sometimes readers used this stance to comment 
on their personal reactions. Thus Robert voiced his reaction to the piece 
as a whole as he finished reading “I See You Never’’: 

That wouldn t be what I would choose to read, but it was pretty good. It’s sad, 
but good. I think I like how it was written. 

Other comments in this stance sought to explain the characters and their 
reactions; Marguerite’s comments reflect this emphasis: 

So, she probably liked the guy. It finally hit her that he’s going away, and she's 
never going to see him again. She probably liked him a lot. 

Jack s comments, on the other hand, pointed to the questions he still had: 

The whole story is very sad. This is an interesting story, but it’s confusing in 
parts. I still don’t know what time period it's in. I still really don't know exactly 
what relationship they have. Obviously she’s just his landlady, I guess. So, I 
don’t know. This is an interesting story. 

Examples from informative texts. The students’ comments revealed a similar 
stance during their readings of “Birth of the Moon.” Susan’s comments 
reflected an attempt to pull the parts together, including the popularity of 
the impactor theory: 

Okay, so everything fits together, so they think. I like that theory too. I can see 
why its their favorite theory, it tells everything, and it really helps you figure out 
what happened with the impactor crash and the chemicals on the earth and the 
moon. 

Marguerite, on the other hand, was more concerned with the language of 
the text, and the problems it could cause: 

That was an interesting article. Confusing though. It was probably for a sci- 
ence magazine or something. But they should have explained some of the vocab- 
ulary a little more. I mean, I knew what they were saying, but maybe not every- 
one would. 

Robert commented briefly on his judgment of the piece and his experi- 
ence: 

Thats it. That was pretty good . I learned something from it. / didnt know 
where the moon might have come from. That was interesting. 

In each of these cases, the students distanced themselves from their en- 
visionment, judged it and its parts, or commented on the meanings they 
had (or had not) made. 

Table 1 provides a summary of the four stances that readers took to- 
ward the texts they were reading. Over time, across the reading of an en- 
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Table 1 



Stances in the Process of Understanding 



Stance 


Focus 


Strategies 


Being Out and 
Stepping Into 
an Envisionment 


gathering 

information 


forms tentative questions 
and associations in attempt 
to build text world 


Being In and 
Moving Through 
an Envisionment 


developing an 
interpretation 


uses local envisionments 
and personal knowledge to 
build and elaborate 
understandings 


Stepping Back and 
Rethinking What 
One Knows 


reflecting on 

personal 

knowledge 


uses growing 
understandings to rethink 
previously held ideas, 
beliefs, or feelings 


Stepping Out and 
Objectifying the 
Experience 


taking a 
critical stance 


distances self from text to 
examine, evaluate, or 
analyze the reading 



experience or aspects of 
the text 



tire piece, each reader wove a growing web of understandings. It was 
woven through the variety of stances the reader took along the way — get- 
ting acquainted, using meaning to build meaning, associating and reflect- 
ing, and finally distancing. Through these shifting relationships between 
self and text, readers structured their own understandings. 



Orientations Toward Meaning 

The stances described so far were an integral part of the process of un- 
derstanding when reading for either literary or informative purposes. 
However, the students’ overall orientation, the particular concerns em- 
phasized and the reasoning strategies that the students used, differed sub- 
stantially in the two contexts. It is these differences that seem to form the 
basic distinctions between reading for literary purposes or for informative 
purposes for the students whom we were studying. The two passages 
chosen as examples in this report also differ sharply in content. However, 
similar differences in orientation, in the way the readers framed their en- 
visionments, were apparent in the other passages used in the study. 

Reaching Toward a Horizon of Possibilities Versus Maintaining a Point of 
Reference 

In any reading, the reader is guided not only by the local envisionment as 
it exists at that point in time, but also by the reader’s sense of the whole. 
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However, the Findings of this study indicate that the role of that overall 
sense of the whole is quite different when the reading takes place in a lit- 
erary as contrasted with informative context. During the reading of liter- 
ature, the sense of the whole changed and developed as the envisionment 
unfolded — it existed as a constantly moving horizon of possibilities (see Iser, 
1978, for discussions of horizon). These possibilities changed and devel- 
oped as the text progressed, emerging out of the developing envision- 
ment of the human situation as reflected in the characters, situations, and 
relationships portrayed in the text. 

In readings of informative text, on the other hand, the sense of the 
whole provided a steady point of reference. As the envisionment unfolded, 
new details in the text clarified the nature of the whole, but they rarely 
changed it. Readers used this constancy in their sense of the whole to 
monitor their understandings (or misunderstandings) of details. Thus 
readers of both literary and informative texts maintained a sense of the 
whole, but the nature of this whole was somewhat different. Reaching to- 
ward a horizon or maintaining a point of reference occurred across the 
reading of each piece — providing an overall orchestration within and 
across the readers’ changing stances. Their understanding of literary texts 
seemed to be constrained by their notions of human possibility, while 
their understanding of informative texts seemed to be constrained by 
their perceptions of the topic. 

Exploring a Horizon of Possibilities 

When readers oriented themselves toward reading for literary purposes, 
their readings were guided by inquisitiveness, by the opening of pos- 
sibilities for the meaning of the whole. Readers in this orientation tried to 
understand what they were reading in terms of its contribution to their 
developing sense of the whole, rather than relating it to an established 
topic or point of view. They clarified ideas as they read and related them 
to the growing and changing horizon of possibilities— that horizon modi- 
fied the parts and the parts modified the horizon. In doing this, readers 
continually tried to open possibilities, see many sides, and go beyond their 
current sense of what the story was about. They considered the underside 
of the story, projecting unspoken emotions and reactions beyond the 
ideas that were more directly expressed. They did this by examining the 
attitudes, assumptions, and intentions reflected in their envisionments. In 
moving toward horizons of possibilities, their focus was always on the 
human situation, on the “vicissitudes of life,” particular or general. As the 
readers made sense of their local envisionments, they did it in a context of 
making sense of the horizon, of the possibilities of the whole, as well. 

Let us look once again at the students’ comments to “I See You Never” 
for evidence of these concerns. Throughout the reading of the literary se- 
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lections, the students attempted to weave the enfolding events into their 
own changing notions of the story as a whole. They treated new events 
not only for their use in momentary understandings, but also as having 
implications for where they believed the story was heading. 

Cara, for example, used Mr. Ramirez’s protests to weave a picture of 
what his prospects would be if he did leave the United States. Her com- 
ments reflect how she thought the story might evolve: 

I look all right, don’t I? And I don’t want to go back. Well, he probably 
doesn’t Want to go back because he’s sort of made himself a good life in the 
United States. And if he ever went to Mexico, he might never be able to immi- 
grate to the U. S. 

Her comments reflect a similar exploration of possibilities when she reads 
Mrs. O’Brian’s lament: 

I just realized . . . I’ll never see him again, Well, Mrs. O’Brian is probably- 
upset because, well, when she described the poor [ Mexican ] landscapes, I can 
sort of see how it will be for him . . . 

Paul also related the ideas he was focusing on to his growing horizon, 
to the possibilities of the whole, as he read: 

One of her sons, behind her, said that her dinner was geting cold, but 
she shook her head at him and turned back to Mr. Ramirez. Right now, 
at a time like this, she wants to point all her attention to Mr. Ramirez because 
she may not be able to see him any more. She doesn’t pay attention to her child 
right now because they will be left, but Mr. Ramirez will be gone. 

As the students focused on the implications of particular ideas for the 
horizon of possibilities in the story, the directions it might lead, they also 
turned each idea they read around, using its potential to fill out the un- 
spoken. They attempted to create a three dimensional world, concerned 
not only with what was said, but also with what was not. Readers thus in- 
vented intentions, motivations, and causalities that went beyond their im- 
mediate text. In doing so, they created a much larger envisionment of the 
text. 

For example, Lesley focused on possible motivations as she read: 

At this table sat Mrs. O’Brian’s children, her three grown sons . . . and 
the two younger daughters were staring at the policemen as they ate. 

So, she must have baked for the policeman. Maybe she’s trying to get Mr. 
Ramirez out of trouble. 

And Paul elaborated on Mrs. O’Brian’s intentions as she commented to 
Mr. Ramirez: 

I’m sorry, Mr. Ramirez, she said, I wish there was something I could 
do. It’s like she knew all the time tlmt he was here illegally, but she was just 
trying to help, I guess. 
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This filling out of possibilities took place at all levels; Lesley was not in- 
terested in motivation, but only with understanding a character’s likes and 
dislikes: 

Pies were baking. Oh, she must like to bake pies. 

Crystal, on the other hand, tried to understand Mrs. O’Brian’s reasons 
for reminiscing about Mexico by projecting the implications for the w r hole 
story: 

. . . with no sound but the waves, no cars . ... So, it seems like she’s try- 
ing to remember what it was like so she can see what he’s going back to. 

In reading literature, then, the students took what they read and used 
it for larger purposes, purposes that went beyond what they were present- 
ly reading, already understood, or had imagined might be. 

Maintaining a Point of Reference 

The process of understanding science and social studies texts was differ- 
ent; with these passages, the students used the content they read to nar- 
row in on increasingly more specific meaning. They began their reading 
trying to gain a notion of the topic or point of the piece, at least in some 
broad sense. This done, throughout the reading they tried to understand 
and clarify what the particular idea they were focusing on meant in terms 
of its relationship to the topic or point of the whole. Although their un- 
derstanding of this topic grew and developed, there was no distant hori- 
zon of possibilities, no questioning of how the point of the whole might 
evolve. Instead, their envisionments contained few ambiguities as they 
narrowed the possibilities of meaning and built a growing web of under- 
standings, all related to the topic which served as their point of reference. 
The process is similar to that which Polanyi (1958) and Koestler (1964), 
among others, have described in examining the nature of creativity in the 
sciences and mathematics: the great breakthroughs have involved the es- 
tablishment of sense of the endpoint or solution, followed by a (sometimes 
difficult) period of developing the detailed solutions or proofs to convey it 
to others. 

The following examples illustrate the students’ attempts to maintain a 
point of reference. When reading “Birth of the Moon,” Robert struggled 
to keep everything focused on the topic as he understood it: 

Like Earth, they contained several different isotopes of oxygen . . . I’m 
wondering what that has to do with the moon. 

Marguerite similarly established the formation of the moon as her stable 
point of reference: 

The mystery of the moon’s birth would soon be solved. They’re trying to 
see how the moon was formed. 
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Crystal’s comments reflected a similar concern with tying the parts to- 
gether cleanly: 

. . . and plunged straight into the center of the Earth. So, this is all just 
supporting the impactor theory. 

In reading for informative purposes, readers were concerned with Clar- 
ifying their envisionments and developing a greater understanding of the 
topic. They focused on connecting informarion — the text’s and their own 
previous knowledge — as a way to make the individual parts cohere and 
their understanding of the topic take shape. When they asked questions, 
the questions were the sort that are topic-related and are generally pro- 
vided by the text— often they anticipate the text — and these questions 
were soon answered. 

Cara continued to tie her understanding of the parts of the text to- 
gether around the topic, consolidating her sense of how earlier parts fit: 

So the moon was left without metallic iron. Well, that’s interesting ’cause 
it sort of explains what they were trying to explain in the beginning of the 
. article. 

Lesley’s comments also reflected his growing understanding of the topic 
as he read: 

A jet pf hot gas squirted 1000 miles into space. So what they’re telling is 
that when it hit the Earth that parts of it clipped to the moon, and that’s why 
they’re alike and that’s why they’re different. 

Thus, reading for literary purposes and reading for informative pur- 
poses appear to be guided by different concerns and involve different 
kinds of thinking; these two different orientations are summarized in 
Table 2. 

The reading of a literary work seems to operate much as Polanyi’s 
(1958) notion of indwelling, where the person lives within the experience, 

Table 2 



Orientations Toward Meaning 



Literary: 


Reaching Toward a 


Readers explore both their local 


Horizon of Possibilities 


envisionments and their overall sense of 
the whole as they enter into and reflect 
upon their text worlds 


Informative: 


Maintaining a 


Readers clarify their ideas and construct 


Point of Reference 


their text worlds by relating what they 
read to their relatively stable sense of the 
topic or point of the piece 
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stretching it in all directions, questioning and reconciling the sometimes 
disparate parts into a coherent and productive whole — readers open pos- 
sibilities and move toward an increased understanding of causes, implica- 
tions, and feelings by moving toward an ever changing horizon that rep- 
resents their sense of how the story might evolve. In contrast, in reading 
for informative purposes they elaborate on their understanding of the 
topic by maintaining a relatively stable point of reference toward which 
they build their understandings. Both orientations allow for creativity and 
the elaboration of new ideas; both are powerful modes of thinking and 
problem solving; but they function in very different ways. 



Discussion 

This study provides a description of the process of understanding when 
reading for literary or informative purposes and suggests the different 
approaches toward meaning readers engage in when reading in these 
ways. When they approach a text, there are four broad stances readers 
adopt to carry them through the experience. These are: “being out and 
stepping into an envisionment,” where readers make the connections nec- 
essary to begin to construct an envisionment; “being in and moving 
through an envisionment,” where they use their envisionments to inform 
their growing understandings; “stepping back and rethinking what one 
knows,” where they use their envisionments to reflect on personal experi- 
ences, ideas, or knowledge; and “stepping out and objectifying the experi- 
ence,” where they look critically at their envisionments, their reading ex- 
periences, and the text itself. Across the reading of an entire piece, 
reading is an act of becoming, where readers use their past experiences, 
the text, and their local envisionments in different ways as they move in 
and out of the various stances as their understandings grow and develop. 

The stances readers took toward the texts they read were shaped in 
part by their initial decision to treat the texts either as literary or infor- 
mative. While readers entered similar stances in each reading experience, 
the focus of their concerns within each stance and thus the ways in which 
they approached the making of meaning differed based upon the overall 
orientation they had chosen. In reading for informative purposes, the stu- 
dents worked closely, using their notions of topic as a frame of reference, 
building and refining meanings as they moved toward a more complete 
understanding of the topic — toward an understanding of what is. How- 
ever, during the reading of literature, they treated their growing under- 
standings more openly, raising possibilities about the horizon as well as 
about their momentary ideas, focusing on the human situation, seeking to 
understand interplays between events and emotions and eventualities — 
toward an understanding of the implications that might be implied in the 
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situation depicted. Both approaches moved the readers to understand, 
but in different ways, toward different meanings. 

Through this lens, we see evidence that when engaged in the tasks set 
for them in this study, the students treated literature as experience and 
not as information. They used literature to invoke insights about the 
human condition, using their own perceptions of what it means to be 
human in the new contexts of what they read. In doing so, they some- 
times contemplated not only the characters and situations embodied in 
the text, but also themselves and the world around them. 

These findings suggest that meaning develops at two levels simul- 
taneously. First, student readers have different assumptions about re- 
sponding to literary versus non-literary texts, and these affect the ways in 
which they orient themselves toward creating their momentary under- 
standings as well as their views of the potential of each piece as a whole. 
At the same time, the similarities in the processes involved in responding 
across the different text types indicate that the four stances represent a 
range of meaning-making options that underlie developing understand- 
ings in general — regardless of text type. This suggests that rather than 
moving “up” a scale of abstraction to an “interpretive” level, students 
learn to develop a store of qualitatively different options to use iii particu- 
lar circumstances for particular purposes. 

Such distinctions are useful in a number of ways. First, the notion of 
stances based on readers’ growing envisionments can provide an alter- 
native to existing category systems that are primarily based on types or lev- 
els of response. Second, the stances are linked to a process of coming to 
understand — a recursive process of approaching and extending the un- 
derstanding of texts. Thus, it would be possible to begin to analyze the ex- 
tent to which readers are or are not willing or able to extend their envi- 
sionments in order to reach a fuller response. It may then also be possible 
to examine reasons underlying readers’ successes or difficulties in extend- 
ing their responses. Considering how students learn to acquire the differ- 
ent stances, as well as the ways in which they engage in the reading of 
literary and other types of texts, may be useful in conceptualizing reader- 
based instructional approaches where both students and teachers can be- 
come more aware of the array of meaning-making options that are avail- 
able in the development of understanding. 

A number of other issues suggested by these findings require further 
study. There is some evidence that although the general orientation a 
reader chooses toward a text is generally maintained, readers occasionally 
shift orientations in quest of meaning. This is particularly interesting in 
that texts often blend techniques (e.g., in the middle of a logical argu- 
ment, the writer may provide a narrative anecdote), thus inviting readers 
to shift their orientations. Schooling may have similar effects. Because in- 
structional materials often treat literature as informative, offering particu- 
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lar content to be learned or interpretations to be memorized, students 
may shift meaning-making approaches based upon their expectations 
about their assignments. Although shifts of these sorts were limited in this 
study, it is possible that more varied text types, longer texts, or more nat- 
ural environments might provide more inviting contexts for such shifts to 
occur. When these occur, by whom, and in response to what texts all need 
to be better understood. 

A final issue suggested by these findings focuses on purposes for read- 
ing and the kinds of texts that are read. For example, different kinds of 
texts as well as different purposes for reading may lead to different pat- 
terns in the way students orient themselves to the text and orchestrate 
their stances. The particular purpose and the particular text, as well as 
the particular kinds of instruction experienced, all are likely to affect the 
strategies students learn to use in academic settings. As Bruner (1986) 
suggests, it is likely that in some instances readers use both orientations, to 
varying degrees, when reading both types of texts for both types of pur- 
poses. Research into this issue will be particularly important as we move 
toward understanding the place of literary understanding in intellectual 
development. 
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Appendix 



Stance 



Jack’s Think-AIoud: “I See You Never” 
(selected sections) 



1 I See You Never. That means someone’s probably never going to see anybody. 
Maybe he’s blind. The soft knock came at the kitchen door. The soft 
knock — which means maybe he’s not a mean person, a soft person. And when 
Mrs. O’Brian opened it, O’Brian, that’s my girlfriend’s name, so I picture my 
girlfriend’s mother. She’s a very nice old lady. There on the back porch were 
her best tenant, Mr. Ramirez, maybe Hispanic or something like that, and 
two police officers. Police officers, that means there might be trouble. One on 
each side of him. Oh, maybe they’re taking him away. Mr. Ramirez just 
stood there, walled in and small. Obviously there’s something going on, be- 
cause maybe Mr. Ramirez got arrested. ... He had come by bus from Mex- 
ico City. He’s Hispanic to San Diego, Okay, so this is taking place in Califor- 
nia. I think San Diego is in California. There he had found the clean little 
room with the glossy blue linoleum and pictures and flowers on the 
flowered walls. Flowered walls, that paints a pretty picture in my mind. And 
Mrs. O’Brian as the strict but kindly landlady. I picture this heavy set 
woman, blond hair, old, caring. You know, almost like a fairy godmother type. 
Strict, neat and tidy, but always willing to give of herself. During the war, 
What war, Korean, Viet Nam, I don’t know when this took place — maybe even 
earlier, he had worked in an airplane factory and made parts for the 
planes that flew off somewhere. Somewhere, that doesn’t give much of an ex- 
planation of where that is. And even now, after the war, he still held this 
job. Okay, so maybe he did something at the airplane factory, in his job. Maybe 
that's why the police officers are there. I don’t know. Mr. Ramirez gazed at his 
feet as if they had carried him into all this trouble. Obviously he’s in trou- 
ble. Maybe it has to do with Mrs. O’Brian. He did something to her that he’s 
ashamed of. “What happened, Mr. Ramirez, asked Mrs. O’Brian?”. . . . 
“I have been here 30 months,” said Mr. Ramirez quietly, He seems like 
he’s ashamed of something, I’m not real sure, looking at Mrs. O’Brian’s 
plump hands. Oh yes, she’s plump, that’s exactly how I pictured her. “That’s 
six months too long,” said one policeman. Six months, that means he should 
have left at 24 months. Right, 30 minus 6 is 24. And 24 months is two years, so 
30 months would be 2’ A years. Okay. 

2 “He only had a temporary visa. We’ve just gotten around to looking for 
him.” Oh, I see, he’s an illegal alien. Okay, I understand. Maybe he’s a drug 
smuggler. No, he wouldn’t be staying at Mrs. O’Brian’s house if he was a drug 
smuggler because she doesn’t like dirty things in the house, I don’t think. He’s ob- 
viously an illegal alien. 
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1 Soon after Mr. Ramirez had arrived, he bought a little radio for his 
room. I don’t know. I don’t understand that. 

2 Evenings he turned it up very loud and enjoyed it. Maybe to get away 
from everything. I mean, she had jive kids and her in the little house. That might 
get a little noisy. Maybe he enjoyed being here. . . . Maybe he didn’t like it where 
he came from. . . . Obviously he didn’t like where he came from, and that’s prob- 
ably why he stayed the 30 months instead of 24. . . . And on many nights he 
had walked silent streets, seeing the bright clothes in the windows, and 
bought some of them. Obviously he’s trying to break out. He's from a lonely 
culture. I see you never. Never, that brings about a thought of loneliness. And 
his room is small, and he's always trying to, maybe he doesn’t like it where he 
came from. Maybe it was real lonely. . . . 

1 “So, here I am,” said Mr. Ramirez, “now to tell you that I must give up 
my room, Mrs. O’Brian.” Why does he have to give up his room? Obviously 
he’s being taken away, so he has to give up his room. But to who? 

3 “I come to get my baggage and clothes and go with these men.” Okay, so 
he’s being forced to leave. These men. Why does he call them men? I hate po- 
licemen. . . . Not that I’ve dealt with them many times in my life, but what 
they’re doing to Mr. Ramirez makes me not trust them. . . . 

1 “Back to Mexico?” “Yes, to Lagos. ...” I wish I had a map here. I don’t 
know where any of these places are. 

2 “I’m sorry, Mr. Ramirez.” 7 guess she probably likes him. Maybe she’s one of 
his lady friends. . . . He says he has a few lady friends. . . . “Okay, here’s the 
key, Mrs. O’Brian,” Mr. Ramirez said. “I have my bag already.” So, he 
gave her the key to his room. . . . He only has one bag and he’s been there for 
2V2 years. My God, I went away for a week last week and I brought four 

bags “You’ve been a good tenant,” said Mrs. O’Brian. A good tenant, 

is that all? “Thank you, Mrs. O’Brian,” he said softly. He closed his eyes. 
He’s probably choked up because he was leaving. He's leaving her and he’s 
choked up about that. He will miss her. . . . One of her sons behind her said 
that her dinner was getting cold. So what. Mr. Ramirez is leaving. That 
makes me mad. The kid is demanding his dinner now. But, he was leaving, and 
they were never going to see him again. . . . But she shook her head at him 
and turned back to Mr. Ramirez. Good for you, Mrs. O’Brian. She re- 
membered a visit she had once made to some Mexican border towns 

. . . the silent towns, the warm beer. The silent towns, silence, loneliness, 
kind of symbolizes Mr. Ramirez. Mr. Ramirez doesn’t say too much. The only 
time he shows affection is when he says thank you. Like “/ have my bag already, 
here’s the key Mrs. O’Brian.” And he still calls her Mrs. O’Brian, which means 
he has respect for her. I’m sure. 
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3 “I don’t want to go back, Mrs. O’Brian,” he said weakly. Weakly , that 
sounds like what I said last week . Last week in Washington I didn't want to 
come back . Now I know why . 'Cause I'm swamped with homework , and I don't 
know — there's so much to do, and my friends are all depressed . . . . 

2 “I like it here. I want to stay here.” That's what I said, that's exactly what I 
said last week and I know what it feels like to go someplace and you like it so 
mush that you don't want to come back . . . . “Mrs. ©’Brian”, he cried sud- 
denly, tears rolling out from under his eyelids. . . . “Mrs. O’Brian, I see 
you never. I see you never.” Does this mean he'll never see her again? ... I 
never heard someone say that . Well, maybe he doesn't speak English that well 
and kind of get the order mixed up . . . . Obviously he likes this lady . Maybe it's 
like a mother to him . . . . She watched them go down the porch steps. . . . 
She must be very sad . She pulled out the chair and sat down. She is sad, I 
can tell . I bet all her five kids feel bad because they want their dinner . And then 
it's cold and they're complaining . They're brats . . . . “What’s wrong, Ma?” 
asked her son. Oh, come on, don't they understand vfhat's going on? Well , 
maybe they don't . ... “I just realized,” said Mrs. O’Brian, “I’ll never see 
Mr. Ramirez again.” Well, that's what he's trying to say, I see you never, 
which is I'll never see Mr. Ramirez again . But they said it in different ways. 

4 The whole story is very sad. This is an interesting story, but it's confusing in 
parts. I still don't know what time period it's in. I still really don't know what 
relationship they have. Obviously she's just his landlady , but I don't know, this 
is an interesting story. 
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ABSTRACT This paper reported a comprehensive 
“meta-review” and synthesis of research on variables related to 
learning, including both cognitive and affective schooling out- 
comes.' A conceptual framework was developed encompassing 
228 items related to school learning, organized a priori into 30 
scales within six categories. Search and selection procedures 
yielded 179 selected handbook and annual review chapters, 
commissioned papers, and other authoritative reviews. Content 
analysis yielded over 3,700 ratings of the strength of influence of 
the variables on learning. The variables confirmed the primacy 
of student, classroom, home, and community influences on 
learning relative to more distal policy variables such as state and 
district characteristics. Additionally, the variables also high- 
lighted the importance of mctacognition, classroom manage- 
ment, quantity of instruction, classroom interactions and cli- 
mate, and the peer group. 



E ducational research lias identified a large number 
of variables related to school learning Because 
such a multiplicity of distinct influences on achievement 
have been found, educators may be perplexed as to which 
items arc most important. Educational researchers, pol- 
icy makers, and practitioners all require clearer guidance 
concerning the relative importance of different learning 
influences and the particular variables most likely to 
maximize school learning To address this need, wc did a 
comprehensive review and synthesis of handbooks, re- 
view annuals, and other highly synthetic prior reviews. 
We characterized the most authoritative scholarly opin- 
ion about ways to optimize educational outcomes across 
a range of educational conditions and settings. This re- 
search synthesis is distinguished by its comprehensive- 
ness, its orientation toward practical school improvement 
strategies, and its focus on comparing the relative contri- 
butions of different items to learning. To organize the 
synthesis. We developed a conceptual framework that 
draws heavily on major theoretical models of school 
learning. Before turning to this framework, we briefly 
describe the evolution of these earlier theoretical models. 



Evolution of Models of School Learning 

J B. Carroll (1963) introduced educational researchers 
to models of school learning in his aptly entitled article, 
“A Mode! of School Learning” in the Teachers College 
Record. In his model, he developed six constructs: apti- 
tude, ability to comprehend instruction, perseverance, 
clarity of instruction, matching the task to student char- 
acteristics, and opportunity to Icam These constructs, 
which succinctly capture the psychological influences on 
school learning, became a point of departure for other 
models to follow; The 1960s and 1970s were marked by 
the introduction of several additional important models 
of learning, including those of Bruner (1966), Bloom 
(1976), Hamischfcgcr and Wiley (1976), Glaser (1976), 
and Bennett (1978). 

All of those models recognized the primary importance 
of student ability, and included constructs such as apti- 
tude, prior knowledge, verbal IQ, and pupil background. 
Most of the models also addressed the importance of mo- 
tivation, by employing such constructs as perseverance, 
self-concept of the learner, and attitude toward school 
subject matter. This acknowledgement of individual dif- 
ference variables among learners contrasted with the 
more narrowly psychological studies of influences on 
learning, which generally treated individual differences as 
a source of error, and focused instead on instructional- 
treatment variables (Hilgard, 1964). 

In addition to student variables, each of the models of 
school learning noted above also gave salience to con- 
structs developed from studies of classroom instruction. 
These constructs varied in generality, some being as 
broad as ‘’instructional events” or “clarity of instruc- 
tion” and others as narrow as ‘‘use of cues” or “feed- 
back and correctives.” 
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Although later models brought some refinement in the 
ways in which individual difference variables and instruc- 
tional variables were defined and the ways in which they 
were related to one another, the primary contributions of 
more recent models have been in extending the range of 
influences considered. Haertel, Walberg, and Weinstein 
(1983), for example, identified nine theoretical constructs 
that exhibit consistent causal influences on academic 
learning: student age or developmental level, ability (in- 
cluding prior achievement), motivation, amount or quan- 
tity of instruction, quality of instruction, psychological 
environment of the classroom, influence of the home, in- 
fluence of the peer group outside of school, and exposure 
to mass media. The researchers showed that previous 
models of school learning neglected extramural and so- 
cial-psychological influences: 

The evolution of models of school learning was further 
advanced with the introduction of models of adaptive in- 
struction (Wang & Lind vail, 1984; Wang & Walberg, 
1985). School-based implementation of models of adap- 
tive instruction are designed to help schools create learn- 
ing environments that maximize each student’s opportu- 
nities for success in school. The models focused on new 
variables associated with instructional delivery systems, 
program design, and implementation. Also, the models 
emphasized those features that Glaser (1982) referred to 
as the “large practical variables” and included efficient 
allocation and use of teacher and student time, a practical 
classroom management system, systematic teacher feed- 
back and reinforcement of student learning behavior and 
progress, instructional interactions based on the diagnosed 
learning needs of individual students, and flexible admin- 
istrative and organizational patterns responsive to pro- 
gram implementation and staffing needs. 

Another contribution to models of school learning 
came from sociologists concerned with the identification 
of effective schools. Edmonds (1979) is most strongly as- 
sociated with this identification of variables associated 
with exceptionally effective schools, especially for the ur- 
ban poor. Significant contributions to effective schools 
models also were made by Brookover (1979), Brookover 
and Lezotte (1979), and Rutter, Maughan, Mortimore, 
Ouston, and Smith (1979).. Illustrations of the types of 
variables characterizing effective schools included degree 
of curriculum articulation and organization, schoolwide 
staff development, parental involvement and support, 
schoolwide recognition of academic success, maximized 
learning time, district support, clear goals and high ex- 
pectations, orderly and disciplined school environment, 
and leadership of principal characterized by attention to 
quality of instruction (Purkey & Smith, 1983). 

Those various models of school learning all contrib- 
uted a variety of items, or variables, that may be useful to 
educational practitioners. Individual researchers may fo- 
cus their work on particular variables or constructs, but 



the purpose of this synthesis was to try to provide a sy- 
noptic view of the entire panoply of variables. 

Methods and Procedures 

The first step in developing the meta-review described 
in this paper was to delineate a comprehensive set of vari- 
ables organized into an inclusive conceptual framework. 
Next, we identified a corpus of over 150 books, book 
chapters, reports, and other sources. The 228 items in the 
conceptual framework were listed on a detailed, 15-page 
coding form, and each of the sources was then coded us- 
ing that form. In all, over 2,500 pages of coding- forms 
were completed. Each citation or discussion of an item 
influencing learning outcomes was coded by page num- 
ber, together with a notation of the reported strength of 
its influence on learning. Those detailed text citations by 
page number were placed in an archive. 2 

The detailed ratings were then recoded onto a set of 
summary forms, one for each chapter or other source, 
which gave overall ratings of strength of influence for 
each of the items discussed in that source. Those sum- 
mary ratings were entered into machine-readable files 
and analyzed to determine the emergent consensus on 
which items exert the most powerful influence on learn- 
ing outcomes. The initial coding tabulated well over 
10,000 separate statements in the research literature con- 
cerning the strength of association between 1 of the 228 
items and learning outcomes. Those statements were re- 
duced to over 3,700 summary ratings, which were then 
keyed and analyzed. 3 

The development of the conceptual framework, selec- 
tion of the corpus of studies, and coding procedures are 
described briefly below. 

Conceptual framework for items related to learning. 
The identification of a comprehensive set of items began 
with a close examination of the models of school learning 
described above, as well as selected sources, including 
Brophy (1986), Keogh, Major-Kingsley, Omori-Gordon, 
and Reid (1982), Wang and Walberg (1985), and Witt- 
rock (1986). Potential variables were written on separate 
index cards, then consolidated and organized into a pre- 
liminary version of the final coding scheme. This draft 
coding scheme was sent to members of the Scientific Ad- 
visory Panel of the Center for Research in Human Devel- 
opment and Education at Temple University. 4 Based on 
detailed commentaries received from the panel members, 
the framework was revised to include four additional 
items and to improve its organization. 

The final framework organized the 228 items related to 
learning into 30 a priori scales within six broad catego- 
ries. The six categories were ordered roughly from more 
distal to more proximal factors. Brief descriptions of the 
categories are presented in Figure 1, together with illus- 
trative items from each scale. 
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Category/Subcategory* Illustrative Variable 



Category l. State and District Variables: These variables are associ- 
ated with state- and district-level school governance and administra- 
tion. They include state curriculum and textbook policies, testing and 
graduation requirements, and teacher licensure, as well as specific 
provisions in teacher contracts, and some district -level administrative 
and fiscal variables. 



District-Level Demographics 
and Marker 
State-Level Policy 



School district size 
Teacher licensure requirements 



Motivational and Affective 
Cognitive 

Metacognitive 

Psychomotor 



Attitude toward subject matter 
instructed 

Level of specific academic 
knowledge in subject area 
instructed 

Comprehension monitoring 
(planning: monitoring effective- 
ness of attempted actions, moni- 
toring outcomes of actions; test- 
ing, revising, and evaluating 
learning strategies) 

Psychomotor skills specific to 
area instructed 



Category 2. Out-of-School Contextual Variables: These variables are 
associated with the home and community contexts within which 
schools function. They include community demographics, peer 
culture, parental support and involvement, and amount of time stu- 
dents spend out of school on such activities as television viewing, lei- 



sure reading, and homework. 
Community 

Peer-Group 

Home Environment and 
Parental Support 
Student Use of Out-of-School- 
Time 



Socioeconomic level of 
community 

Level of peers’ academic 
aspirations 

Parental involvement in assuring 
completion of homework 
Student participation in clubs 
and extracurricular school 
activities 



Category 3. School-Level Variables : These variables are associated 
with school-level demographics, culture, climate, policies, and prac- 
tices. They include demographics of the student body, whether the 
school is public or private, and levels of funding for specific categori- 
cal programs; school-level decision-making variables, and specific 
school-level policies and practices, including policies on parental in- 
volvement in the school. 



Demographic and Marker 
Teacher/Administrator Decision- 
Making 

School Culture (Ethos conducive 
to teaching and learning) 

School-Wide Policy and 
Organization 
Accessibility 



Parental Involvement Policy 



Size of school 

Principal actively concerned with 
instructional program 
School-wide emphasis on and 
recognition of academic 
achievement 

Explicit school-wide discipline 
policy 

Accessibility of educational 
program (overcoming architec- 
tural, communication, and envi- 
ronmental barriers) 

Parental involvement in im- 
provement and operation of 
instructional program 



Category 4. Student Variables: These variables are associated with in- 
dividual students, including demographics, academic history, and a 
variety of social, behavioral, motivational, cognitive, and affective 
characteristics. 

Demographic and Marker Gender 

History of Educational Prior grade retentions 

Placement 

Social and Behavioral Positive, nondismptive behavior 

Figure 1. Conceptual Framework With Illustrative Examples 



Category 5. Program Design Variables: These variables are associated 
with instruction as designed and with the physical arrangements for its 
delivery. They include the instructional strategies specified by the cur- 
riculum and characteristics of instructional materials. 

Demographic and Marker Size of instructional group 

(whole class, small group, one- 
on-one instruction) 

Curriculum and Instructional Alignment among goals, con- 
tents, instruction, assignments, 
and evaluation 

Curriculum Design Materials employ advance 

organizers 



Category 6. Implementation, Classroom Instruction, and Climate 
Variables: These variables are associated with the implementation of 
the curriculum and the instructional program. They include classroom 
routines and practices, characteristics of instruction as delivered, 
classroom management, monitoring of student progress, and quality 
and quantity of instruction provided, as well as student-teacher inter- 
actions and classroom climate. 



Classroom Implementation 
Support 

Classroom Instructional 
Quantity of Instruction 

Classroom Assessment 

Classroom Management 



Student and Teacher Inter- 
actions: Social 

Student and Teacher Inter- 
actions: Academic 

Classroom climate 



Establishing efficient classroom 

routines and communicating 

rules and procedures 

Use of clear and organized direct 

instruction 

Time on task (amount of time 
students are actively engaged in 
learning) 

Use of assessment as a frequent, 
integral component of instruc- 
tion 

Group alerting (teacher uses 
questioning/ recitation strategies 
that maintain active participation 
by all students) 

Student responds positively to 
questions from other students 
and from teacher 
Frequent calls for extended, sub- 
stantive oral and written re- 
sponse (not one-word answers) 
Cohesiveness (members of class 
are friends sharing common 
interests and values and empha- 
sizing cooperative goals) 



*Subcategories are listed below the description of each broad category and are 
each illustrated with repre sen tative variables. For example, the first broad cate- 
gory includes two subcategories, ’’District -Level Demographics and Marker Var- 
iables” and "State- Level Policy Variables.” 
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Selection of a corpus of studies. A vast research litera- 
ture addresses one or more of the potential learning influ- 
ences represented by the conceptual framework, and one 
could not possibily examine all of the thousands of origi- 
nal studies relevant to a synthesis of this scope. Even the 
literature of review articles is massive. For this reason, we 
focused on authoritative reviews and handbook chapters, 
especially those sponsored by the American Educational 
Research Association and other organizations, and we se- 
lected additional syntheses in government documents and 
other sources. A preliminary list of sources was reviewed 
by the Scientific Advisory Panel and revised following 
their recommendations. Folk *ving this review, the 
sources chosen included chapr s from the past decade or 
more of the Review of Research in Education , the An- 
nual Review of Psychology , and the Annual Review of 
Sociology , as well as the Handbook of Research on 
Teaching (Wittrock, 1986), Designs for Compensatory 
Education (Williams, Richmond, & Mason, 1986), more 
specialized handbooks, and a small number of journal 
articles chosen to assure coverage of all of the areas ad- 
dressed in the comprehensive framework. Initially, over 
200 articles, chapters, and other sources were identified. 
All of those sources were read, but some were excluded 
from the final corpus because they failed to address kin- 
dergarten through grade 12 instruction in regular school 
settings, because they addressed exceptionally narrow 
and atypical learning outcomes, or because they were 
relevant only to rare or special-learner populations. 

A total of 179 sources were included in the final corpus 
of studies (see Appendix for a complete bibliography). 
All of those sources were relevant to a range of cognitive 
and affective learning outcomes for kindergarten through 
grade 12 learners in formal educational settings. Table 1 
presents a summary by type of the source documents in- 
cluded in the final synthesis. 

Coding procedures. Each source document was coded 
initially onto a detailed rating form, which allowed the 
recording of multiple references in a single document to 
the same item. In addition to coding references to the 228 
prespecified items, space was provided for the coding of 
any additional items related to learning outcomes, referred 



Table 1. — Number and Percentage of Source Documents by type 



Type of source 1 


N 




Total pages 


Chapters for annual review series 


86 


48 


3,179 


Handbook chapters 


44 


25 


1,089 


Government documents and 


20 


11 


772 


commissioned reports 
Book chapters 


18 


10 


563 


Review articles in journals 


11 


6 


152 


Total 


179 


100 


5,755 



*A complete bibliography is available from the first author. 



to on the form as supplementary items. Brief notes also 
were recorded for most sources, including page refer- 
ences, comments on the source’s overall relevance, and 
any limitations on the learner populations or varieties of 
learning outcomes addressed. This archived documenta- 
tion has been retained by the first author. 

Each reference to an item’s relation to learning out- 
comes was coded on a 3-point scale, with 1 representing a 
weak, uncertain, or inconsistent relation to learning, 2 
representing a moderate relation, and 3 representing a 
strong relation. Where “vote counts” or proportions of 
confirming studies were reported, a 3 indicated that more 
than 80% of the studies discussed had found a statistically 
significant association of an item to achievement, a 2 in- 
dicated that between 40 % and 80% of the studies found 
support for the relationship, and a 1 indicated less than 
40% in support. Where results were summarized in terms 
of effect sizes, we assigned a code of 3 to effect sizes 
greater than .33, 2 to effect sizes of .10 to .33, and 1 for 
smaller effect sizes. Where correlations were reported, we 
used 3 for correlations greater than .40, 2 for correlations 
of .15 to .40, and 1 otherwise. 

In many cases, the source documents did not present 
quantitative indices like effect sizes or correlations, so we 
had to judge the strength of the evidence presented from 
prose descriptions of the conclusions from bodies of re- 
search. In those cases, the strength of the evidence pre- 
sented was judged weak, moderate, of strong, and coded 
accordingly. Even though all of the 228 items were defined 
in such a way that they were expected to relate positively 
to learning, in rare instances the literature reported nega- 
tive conclusions. 5 

Following the coding of all specific references by page 
number, we transcribed ratings onto a second summary 
form for each source, prior to keying for data analysis. 
At this stage, a single summary code— the average of all 
the ratings for each source document— was recorded in- 
dicating the strength of association for each item discussed 
in the source, according to the preponderance of the spe- 
cific references noted. 6 

Data Analysis 

After inspecting univariate frequency distributions for 
each of the 228 separate items (o assure that no values 
were out of range, we aggregated the separate items to 
the level of the 30 scales described in Figure 1. We accom- 
plished this procedure by taking the average of all non- 
missing values in a scale, for each source. In cases where 
a source document did not discuss any of the separate 
items in a scale, we entered a missing data code. In those 
rare cases where negative findings were coded, we retained 
their negative signs when averages were taken. 

In a second stage of data reduction, six additional vari- 
ables were created corresponding to the categories de- 
scribed in Figure 1. The values of those variables for each 
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source were weighted averages of all nonmissing scale 
values comprising that category. 7 Means, standard devia- 
tions, and alpha reliabilities for the six categories and 30 
scales are presented in Table 2. The reliabilities for docu- 
ments (not raters) ranged from .71 to .99; all but four 
were greater than .80 and most exceeded .90.® 

Table 2 also reports the number of sources that dis- 
cussed items in each scale. Surprisingly, the frequency 
with which different scale items are discussed in the liter- 
ature was not a reliable guide to their importance for 
learning outcomes. The Spearman rank correlation be- 
tween frequencies and means across the 30 scales was 
only .10. 



Summary of Results 

Table 2 presents the importance of many distinct influ- 
ences on school learning outcomes. Over all 30 scales, the 
mean rating was roughly 1.8, a little below the level desig- 
nated “moderate relation to learning. “ More important, 
however, the synthesis shows which categories, scales, 
and specific items are most strongly associated with 
leamng outcomes. In discussing results by category and 
by scale, we presented relevant findings concerning 
specific items to clarify or elaborate the category- and 
scale-level findings reported in Table 2. 

At the highest level of generality, this synthesis con- 
firmed the importance of the quality of schooling for 



Table 2. — Reliabilities, Means, Standard Deviations, and Frequencies for Source Ratings 



Category/subcategory 


Reliability* 


M 


SD 


Frequency 


State and district variables 


.90 


1.22 


.81 


27 


District demographics and marker 


.95 


1.46 


.50 


14 


State-level policy 


N.C. 


1.24 


1.00 


19 


Out-of-school contextual variables 


.99 


1.87 


.39 


59 


Community 


N.C. 


1.80 


.41 


15 


Peer-group 


.98 


2.00 


.34 


18 


Home environment and parental support 


.95 


1.90 


.40 


47 


Student use of out-of-school time 


N.C. 


1.94 


.46 


17 


School-level variables 


.95 


1.54 


.96 


102 


Demographics and marker 


.91 


1.74 


.56 


25 


Teach cr/administ rat or decision making 


.87 


1.65 


.95 


21 


School culture 


.87 


1.84 


.43 


49 


School-wide policies and organization 


.76 


1.40 


1.14 


74 


Accessibility 


N.C. 


2.00 


.00 


2 


Parental involvement policy 


N.C. 


1.67 


.56 


23 


Student variables 


.92 


1.83 


.57 


155 


Demographics and marker 


.71 


1.70 


.77 


90 


History of educational placements 


N.C. 


0.16 


1.80 


19 


Social and behavioral 


.80 


1.98 


.34 


35 


Motivational and affective 


.91 


1.93 


.42 


81 


Cognitive 


.88 


1.98 


.33 


101 


Metacognitive 


.91 


2.08 


.36 


76 


Psychomotor 


N.C. 


2.33 


.52 


6 


Program design variables 


.90 


1.90 


.38 


142 


Demographic and marker 


N.C. 


1.97 


.54 


23 


Curriculum and instruction 


.90 


1.92 


.46 


108 


Curriculum design 


.89 


1.88 


.34 


97 


Classroom instruction and climate variables 


.97 


1.84 


.66 


165 


Classroom implementation support 


.85 


1.84 


.38 


66 


Classroom instructional 


.89 


1.85 


.74 


156 


Quantity-of-inst ruction 


.94 


2.02 


.64 


69 


Classroom assessment 


N.C. 


1.89 


.30 


61 


Classroom management 


.98 


2.07 


.23 


42 


Student/teacher interactions: social 


.73 


2.02 


.41 


44 


Student/teacher interactions: academic 


.77 


1.89 


.44 


29 


Classroom dimate 


.99 


2.01 


.38 


75 



'Coefficient alpha reliabilities were estimated for each scale from average variances and interitem covariances. 
Due to missing data, ratings for some cases are based on fewer items. Thus, obtained reliabilities are somewhat 
lower than the figures reported in this table. N.C. indicates values that were not calculable, either because scales 
consisted of only a single item or because of patterns of missing data. 
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learning outcomes. Of the six categories, the highest rat- 
ings overall were assigned to Program Design Variables, 
followed by Out-of-School Contextual Variables. The 
category reflecting the quality of instruction as delivered. 
Classroom Instruction and Climate Variables, ranked 
third in importance, closely followed by Student Vari- 
ables. The last two categories, School-Level Variables 
and State and District Variables, received markedly lower 
ratings overall. This overall ranking of sources of influ- 
ence contrasted sharply with the “conventional wisdom** 
because the Equality of Educational Opportunity (EEO) 
Survey (Coleman et al., 1966) stated that quality of 
schooling has relatively little impact on schooling out- 
comes relative to out-of-school socioeconomic variables. 

The importance of proximal psychological variables 
may be seen in the scales /that obtained the highest 
ratings. 9 Those scales with mean ratings of 2.00 or greater 
(beginning with the highest) were: 



Metacognition 


X = 2.08 


Classroom management 


X = 2.07 


Quantity of instruction 


X = 2.02 


Student/teacher interactions: Social 


X = 2.02 


Classroom climate 


X = 2.01 


Peer group influences 


X « 2.00 


In the remainder of this section, 


the categories and 



scales are discussed in turn, and those scales and items 
that received exceptionally high ratings are highlighted. 
The categories representing instruction as designed and 
instruction as delivered are discussed first. Those cate- 
gories are followed by out-of-school context and student 
characteristics. Finally, the more distal variable cate- 
gories of school level variables and state and district vari- 
ables are addressed. 

Program Design Variables 

This category includes instruction as designed, and the 
physical arrangements for its delivery, organized into 
three scales, as shown in Figure 1 and Table 2. The scale 
Demographic and Marker Variables was rated highest of 
the three, and, within this scale, the most highly rated 
items are size of instructional group (whole class, small 
group, or one-on-one instruction), number of classroom 
aides, and resources needed. (Ratings for those items 
ranged from 1.95 to 2.00). Thus, the most important as- 
pect of program design appeared to be the intensity of 
educational services provided to each learner. More 
aides, smaller groups, or increased material resources 
were associated with significantly higher learning out- 
comes. 

Curriculum and Instructional Variables includes items 
with average ratings above 2.0 (moderate relation to 
teaming). The highest rated of those suggest that the key 
to effective instructional design is the flexible and appro- 
priate use of a variety of instructional strategies, while 



maintaining an orderly classroom environment. The 
highest overall rating in this scale was for use of . . . tech- 
niques to control classroom disruptiveness. This item was 
followed by use of prescriptive instruction combined with 
aspects of informal or open education and presence of in- 
formation in the curriculum on individual differences 
and commonalities, both of which explicitly relate to stu- 
dent diversity and individualization. Other highly rated 
items referred to specific instructional strategies, includ- 
ing use of mastery learning techniques, . . . instructional 
cues, engagement, and corrective feedback ... use of co- 
operative learning strategies, and use of diagnostic-pre- 
scriptive methods. 

Curriculum Design also includes several items* with 
average ratings near 2.0, although none exceeds the mod- 
erate level. High ratings were given to, materials employ 
alternative modes of representation and degree of struc- 
ture in curriculum accommodates needs of different 
learners, both of which reinforce the importance of of- 
fering a variety of instructional materials and approaches 
to accommodate individual differences. The importance 
of the organization of curriculum content is revealed by 
two highest rated items in this scale, materials employ 
learning hierarchies and material is presented in a cogni- 
tively efficient manner. 

Implementation , Classroom Instruction , and 
Climate Variables 

This category includes support of the curriculum and 
the instructional program, classroom routines, specific 
instructional, assessment, and classroom management 
practices; quantity of instruction, academic and nonaca- 
demic student-teacher interaction, and classroom cli- 
mate. By far the largest of the six categories, this group 
constitutes 79 of the 228 items and 8 of the 30 scales. Half 
of those scales had mean ratings above 2.00, placing 
them overall among the most influential scales. 

High ratings in the areas of implementation, classroom 
instruction, and climate again highlight the importance 
of maintaining an orderly classroom environment and 
providing clear, well-organized instruction appropriate to 
the needs of individual learners. In the overall ranking of 
all 30 scales. Classroom Management ranked second, and 
its most critical items were group' alerting (teacher uses 
questioning/recitation strategies that maintain active par- 
ticipation by all students) and learner accountability 
(teacher maintains student awareness of learning goals 
and expectations). Smooth transitions from one instruc- 
tional activity to another, minimal disruptions, and 
teacher awareness of classroom activity at all times also 
received mean ratings above 2.00. 

Quantity of Instruction was ranked third overall, fol- 
lowing Classroom Management. This included scale time 
spent in direct instruction, especially direct instruction on 
basic skills, time spent on homework, and length of the 
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school day and year. The importance accorded quantity 
of instruction is not surprising. This construct has ap- 
peared in many of the most widely cited models of school 
learning (Haertel, Walberg, & Weinstein, 1983). 

Student /teacher interactions. Social ranked fourth 
overall, and Classroom Climate was ranked fifth. The 
high ranking for social interactions was almost entirely 
caused by two items with mean ratings of 2.00 or greater: 
teacher reacts appropriately to correct and incorrect an- 
swers and student responds positively to questions from 
other students and from teacher. 10 Classroom climate in- 
cluded 15 items with ratings of 2.00 or greater. Together, 
the highly rated items in those two scales characterize a 
classroom in which teacher and students interact consid- 
erately and cooperatively, where students work with sev- 
eral classmates, share common interests and values, and 
pursue cooperative goals. In that type of classroom, stu- 
dents are actively engaged in learning and are involved in 
making some types of classroom decisions. Concurrently, 
the class is well organized and well planned, with a clear 
academic focus. Objectives of learning activities are spe- 
cific and explicit, and students feel continually and ap- 
propriately challenged, with the pacing of instruction ap- 
propriate for the majority. 

The remaining scales under Implementation, Class- 
room Instruction, and Climate Variables had much lower 
overall ratings, but they included more than 20 specific 
items with means of 2.00 or greater. The majority of 
those items referred to instructional organization and to 
mechanisms for assuring that students understand that 
organization and the goals of instruction. For example, 
high ratings were given to the use of advance organizers 
and directing students’ attention to the content to be 
learned, as well as to clear and organized direct instruc- 
tion, systematic sequencing of lesson events, and clear 
lesson transitions. Other highly rated items included cor- 
rective feedback in case of student error, frequent aca- 
demic questions, and accurate measurement of skills. Fi- 
nally, the literature strongly supported the teaching of 
skills in the context of meaningful applications, use of 
good examples and analogies, and teaching for meaning- 
ful understanding, together with explicit promotion of 
student self-monitoring of comprehension and gradual 
transfer of responsibility for learning from the teacher to 
the student. 

Extramural Variables 

This category included items associated with the home 
and community contexts within which schools function. 
As presented above, Peer-Group Variables was ranked 
sixth among all scales. This ranking was primarily caused 
by the emphasis placed on peers* educational and occu- 
pational aspirations, both of which had mean ratings of 
2.00 or higher. 

Additional highly rated items in this category reflected 
parental interest and involvement in students* school- 



work. For example, parental involvement in assuring 
completion of homework, parental participation in 
school conferences and related activities, and parental in- 
terest in students’ school work all received high ratings. 
The educational environment of the home (e.g., number 
of books and magazines) also was cited in numerous 
sources and received consistently high ratings. Student 
participation in clubs and extracurricular school activities 
and time spent on leisure reading also were moderately 
related to learning outcomes. 

Student Variables 

Student Variables items were associated with in- 
dividual students themselves, including demographics, 
academic history, and various social, cognitive, and af- 
fective characteristics. Among those items, psychomotor 
skills specific to area instructed had the highest rating, 
which was 2.33. This item was the only one included in 
the Psychomotor Variables scale. However, as explained 
above, this mean was based on only six sources. This 
mean was best regarded as a statistical artifact and will 
not be further discussed. 

Metacognitive Variables received the highest mean rat- 
ings of any of the remaining scales in the entire frame- 
work. Highly rated metacognitive items included com- 
prehension monitoring (planning; monitoring effective- 
ness of attempted actions; testing, revising, and evaluat- 
ing learning strategies), self-regulatory, self-control strat- 
egies (e.g., control of attention), and positive strategies to 
facilitate generalization of concepts. 

Several specific items in the remaining Student Vari- 
ables scales also had high ratings, including positive be- 
havior and ability to make friends with peers, motivation 
for continual learning, annd perseverance on learning 
tasks. Highly rated items from the Cognitive scale included 
several representing general mental abilities, levels of ba- 
sic skills sufficient to profit from instruction, and prior 
knowledge in the subject area instructed. 

School-Level and State and District Variables 

Educational policy items at the school, district, and 
state levels appeared from this research synthesis to have 
relatively little association with learning outcomes, as 
shown by low mean ratings for categories and scales. A 
few items in this area received mean ratings of 2.00 or 
higher, but nearly all of those were based on fewer than 
10 sources. Nonetheless, several school-level educational 
practices emerged as important. Those practices included 
the presence of an effective schools program, explicit 
school grading, academic progress, and attendance poli- 
cies, and a safe and orderly school climate. Peer and 
cross-age tutoring, which were classified as school-level 
variables when their implementation required coordina- 
tion among self-contained classrooms, also received 
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moderate or higher ratings based on discussions in more 
than 10 sources. 

Discussion 

This research synthesis confirms that distal policy vari- 
ables are less important to schooling outcomes than 
quantity and quality of instruction, home environment, 
or student characteristics. Of the six categories in the con- 
ceptual framework (see Figure 1 and Table 2), State and 
District Variables and School-Level Variables, both com- 
prising mainly policy variables, had markedly lower 
mean ratings than the remaining four categories. The 
items most important to learning outcomes were those 
that were directly tied to students’ engagement with the 
material to be learned. 

In contrast to the earlier view that quality of schooling 
is of little importance relative to out-of-school factors 
(Coleman et al., 1966), our synthesis also suggests that 
from kindergarten through Grade 12, across a range of 
content areas and educational contexts, quality and quan- 
tity of instruction are roughly equal in importance to stu- 
dent characteristics and out-of-school contextual items. 

Furthermore, the present synthesis of educational re- 
search is considerably more comprehensive than What 
Works: Research About Teaching and Learning , the 
widely distributed pamphlet of the U.S. Department of 
Education (1986), and it contains both highly effective 
and relatively less-effective practices. The present synthe- 
sis, moreover, draws on a larger body of literature and 
contains a more explicit methodology that can be repli- 
cated by other investigators. This research contains some 
228 practices in comparison with 41 in the original What 
Works (and 62 in the second edition), and it gives a nu- 
merical rating to each one as well as composites. Yet 
none of the findings of What Works and the present 
work are discordant. What Works contains specific find- 
ings and elaborates on and illustrates various techniques. 
Such techniques are described specifically enough to be 
understood by parents and teachers; perhaps they might 
even be put into practice without assistance. 

Regarding the 30 scales, those identified as most im- 
portant to good learning outcomes are student metacog- 
nition, effective classroom management, quantity of in- 
struction, positive and productive student/teacher inter- 
actions, a classroom climate conducive to learning, and a 
peer culture supportive of academic achievement. Those 
broad conclusions are supported by more specific find- 
ings from the research synthesis. Those selected findings 
are highlighted below. 

Student characteristics . Individual differences among 
students have long been recognized as critical determi- 
nants of learning outcomes, but we were surprised and 
encouraged that in this synthesis the metacognitive items 
emerged as most important, including comprehension 
monitoring, use of self-regulatory, self-control strategies. 



and use of strategies to facilitate generalization of con- 
cepts. Metacognitive variables are heavily cited in the cur- 
rent literature, in contrast to an earlier focus on relatively 
stable general mental abilities. A better understanding of 
those alterable variables may ultimately help the great 
majority of students to reach higher achievement levels 
through appropriate training in metacognition. Two ad- 
ditional student items accorded importance in the re- 
search literature are perseverance on learning tasks and 
motivation for continual learning. Both of those factors 
reinforce the conclusion that consistent engagement with 
the subject matter to be learned is critical to school success. 

Quality and quantity of instruction . Classroom man- 
agement and climate and student-teacher interactions 
represent an important constellation of variables related 
to effective instruction. Detailed examination of the 
highly rated items in those areas reveals a portrait of co- 
operative, cohesive, goal-directed classrooms, in which a 
variety of educational approaches and activities are em- 
ployed. Items heavily cited in the research literature in- 
clude sound organization and systematic sequencing of 
instruction and effective use of direct, teacher-centered 
instruction. Among other instructional approaches fre- 
quently linked to positive learning outcomes are peer and 
cross-age tutoring and cooperative group-learning strat- 
egies. 

Several items associated with quantity of instruction 
also emerged as important, including student time on 
task, length of school day and year, amount of time allo- 
cated to direct instruction in basic skills, and time spent 
out of school on homework and on leisure reading. Of 
these variables, the most frequently cited one is time on 
task. These time-related variables have clearly become 
well established and widely accepted as determinants of 
learning outcomes, in spite of criticims cited by Shulman 
(1986) of time as an “empty vessel.” 

Out of school context. Researchers are giving increas- 
ing attention in the research literature to the role of pa- 
rental involvement and support variables in promoting 
student learning. The synthesis affirms the importance of 
those hems, as well as peer-group influences. Those find- 
ings are reflected in ratings for parental involvement in 
school activities, interest in schoolwork, and monitoring 
of school attendance and homework completion. Paren- 
tal support might also be mediated through influence on 
students' selection of friends. Peer group variables, espe- 
cially academic and occupational aspirations, are strongly 
related to school success. 

Strength of influences on school learning. Physical 
processes can often be explained as functions of a small 
number of variables interacting in simple ways. In con- 
trast, schooling processes respond to a multitude of influ- 
ences interacting in kaleidoscopic patterns. This research 
synthesis has confirmed that a large number of variables 
are moderately related to learning outcomes, but few, if 
any, single variables are strongly related to learning. Au- 



National Library of Education 



94 



98 



||f Subject-Area Learning 



Journal of Educational Research 



thors of original research studies and of reviews and syn- 
theses are appropriately cautious in stating the impor- 
tance of particular items, and their caution is reflected in 
the relatively narrow range of mean ratings presented in 
Table 2. Nonetheless, taken together, the items examined 
in this synthesis are powerful determinants of school ef- 
fects. 

NOTES 

1. This research was supported in pan by the Temple University Cen- 
ter for Research in Human Development and Education and in pan 
by a grant from the U.S. Department of Education’s Office of Spe- 
cial Education and Rehabilitative Services. The opinions expressed 
herein are solely those of the authors, and no official endorsement 
should be inferred. 

2. Copies of the detailed coding form and complete bibliographic cita- 
tions for the 179 sources, as well as copies of the data archive, are 
available from Margaret C. Wang, Center for Research in Human 
Development and Education, Ritter Hall Annex, 9th Floor, Temple 
University, Philadelphia, PA 19122. 

3. In addition to the coding and analysis of the 179 source documents, 
a survey also was conducted of the authors of all major source doc- 
uments examined. The summary coding form described below was 
distributed to authors, with a request to provide overall ratings of 
the importance of the 228 items to learning outcomes. A total of 78 
forms were returned. Those expert ratings were analyzed separately 
from the source document ratings, following identical procedures. 
Results were highly similar, with the exception that the experts gen- 
erally tended to give somewhat higher numerical ratings. 

4. This panel included 12 prominent experts in areas of research on 
trarhing , education, educational psychology, and special education. 

5. Most of those occurred for items in the scale History of Educational 
Placements, which accounts for the low mean of this variable in 
Table 2. 

6. If any supplementary items had been coded, those were reexamined 

as the forms were transcribed and, whenever possible, were included 
under one of the prespecified items. This procedure was generally 
possible because most supplementary items documented authors* 
more or specific empirical conclusions, for example, spe- 

cific types of motivation related to teaming or particular variants of 
instructional practices. Such detailed findings were incorporated 
into the broader variable prespcdfied on the form. The other sup- 
plementary items were those documenting two-way or occasionally 
higher order interactions. Because interactions rep r e se nt more sub- 
tle findings and frequently fail to replicate, they were not trans- 
ferred from the detailed form to the summary form. The summary 
forms were keyed and verified, and fiks were prepared for data 
analysis using standard statistical software pac k ages. 

7. The weights used were equal to the numbers of original items in- 
cluded in the respective scales. Note that if there were no m iss in g 
data, this procedure would result in giving all of the original items in 
a broad category equal weight. Where some items in a scale are 
missing, this procedure in effect assigns the mean of the nonmissing 
scale items to those missing observations. For any given scale, about 
15% of the values of items on average were missing. 

8. As noted in the footnote to the table, those reported reliabilities are 
for means of aD the items in a given category or scale. Due to miss- 
ing data, values for some sources were based on means of fewer 
items. 

9. The highest ratings overall were assigned to Psychomotor Vari- 
ables and a moderately high rating also was assigned to the scale Ac- 
cessibility Variables. However, only one item was included in each 
of those scales, and those items were referred to in 6 or fewer of the 
179 sources analyzed. Thus, Psychomotor Variables and Accessibil- 
ity Variables were set aside. The list of scales with the highest ratings 
include the 28 scales with more items and ratings. 

10. A third item in this scale, teacher provides explicit coaching to re- 
duce aggression, also received a mean rating above 2.00 but was 
mentioned in only 4 of the 179 sources. That item is of limited rele- 
vance in most regular educational settings. 



APPENDIX 

Bibliographic References for the 179 Sources 
Synthesized in This Article 



Adams, M. J. (1986). Teaching thinking to Chapter 1 students. In B. 1. 
Williams, P. A. Richmond, A B. J. Mason (Eds.), Designs /or com- 
pensatory education : Conference proceedings and papers (pp. 1V-85 
to IV-119). Washington, DC: Research and Evaluation Associates. 

Algozzine, R., A Mercer, C. D. (1980). Labels and expectancies for 
handicapped children and youth. In L. Mann A D. A. Sabatino 
(Eds.), Fourth review of special education (pp. 287-313). New York: 
Grune A Stratton. 

ADington, R. L., A Johnston, P. (1986). The coordination among regu- 
lar classroom reading programs and targeted support programs. In B. 
1. Williams, P. A. Richmond, A B. J. Mason (Eds.), Designs for 
compensatory education : Conference proceedings and papers (pp. 
VI-3 to VI-40). Washington. DC: Research arid Evaluation 
Associates. 

Alter, M., A Gottlieb. J. (1987). Educating for social skills. In J. Gott- 
lieb A B. W. Gottlieb (Eds.), Advances in special education (Vol. 6, 
pp. 1-61). Greenwich, CT: JAI Press. 

Ambramson, M. (1980). Implications of mainstreaming: A challenge 
for special education. In L. Mann A D. A. Sabatino (Eds.), Fourth re- 
view of special education (pp. 315-340). New York: Grune A Stratton. 

Anderson, L. (1985). A retrospective and prospective view of Bloom's 
"Learning for Mastery.” In M. C. Wang A H. J. Walberg (Eds.), 
Adapting instruction to individual differences (pp. 254-268). 
Berkeley, CA: McCutchan. 

Archarabauh, F. X., Jr. (1986). Instructional setting: Key issue and bo- 
gus concern. In B. 1. Williams, P. A. Richmond, A B. J. Mason, 
(Eds.), Designs for compensatory education: Conference proceedings 
and papers (pp. 111-59 to 111-93). Washington, DC: Research and 
Evaluation Associates. 

Annento, B. J. (1986). Research on teaching social studies. In M. C. 
Wittrock (Ed.), Handbook of research on teaching (3rd cd., pp. 
942-951). New York: Macmillan. 

Baldwin, A. L. (1973). Social teaming. In F. N. Keriinger (Ed.), Review 
of resear ch in education (Vol. 1, pp. 34-57). Itasca, 1L: Peacock. 

Barr, R., A Dreeban, R. (1977). Instruction in classrooms. In L. S. 
Shutman (Ed.), Review of research in education (Vol. 5, pp. 89-162). 
Itasca, 1L: Peacock. 

Bennett, W. J. (1986). First lessons. A report on elementary education 
in America. Washington. DC: U.S. Department of Education. 

Bereiter, C., A Scardamalia, M. (1986). Levels of inquiry into the nature 
of e xpert ise in writing. In E. Z. Rothkopf (Ed.), Review of research in 
education (Vol. 13, pp. 259-282). Washington, DC: American Edu- 
cational Research Association. 

Berliner, D. C., A Cahen, L. S. (1973). Trait-treatment interaction and 
learning. In F. N. Keriinger (Ed.), Review of research in education 
(Vol. 1, pp. 58-94). Itasca. IL: Peacock. 

Biddle, B., A Anderson, D. S. (1986). Theory, methods, knowledge, 
and research on teaching. In M. C. Wittrock (Ed.), Handbook of re- 
search on teaching (3rd cd., pp. 230-252). New York: Macmillan. * 

Blackburn, A. E., Bon, D. A., A Cross, D. P. (1987). Noncat ego rical 
special education personnel preparation. In M. C. Wang, M. C. Rey- 
nolds, A H. J. Walberg (Eds.), Handbook of special education: Re- 
search and practice. Volume l: Learner ch ar acte ris tics and adaptive 
education (pp. 313-329). Oxford: Pergamon. 

Blanton, L. P., Sitko, M. C., A Gillespie, P. H. (1976). Reading and 
the mildly retarded: Review of research and implications. In L. Mann 
A D. A. Sabatino (Eds.), Third review of special education (pp. 
143-162). New York: Grune A Stratton. 

Block, J. H., A Bums, R. B. (1975). Mastery teaming. In L. S. Shulman 
(Ed.), Review of research in education (Vol. 4, pp. 3-49). Itasca, 1L: 
Peacock. 

Brainin, S. S. (1985). Mediating teaming: Pedagogic issues in the im- 
provement of cognitive functioning. In E. W. Gordon (Ed.), Review 
of research in education (Vol. 12, 121-155). Washington, DC: Amer- 
ican Educational Research Association. 

Brantlingr, E. A., A Guskin, S. L. (1987). Ethnocultural and social-psy- 
chological effects on teaming characteristics of handicapped children. 




BEST COPY AVAILABLE 
q< 99 



95 



Volume 2 y Spring 1997 



Subject-Area {Learning 



September/October 1990 [Vol. 84<No. 1)1 



In M. C. Wang, M. C. Reynolds, & H. J. Walberg (Eds.). Handbook 
of special education: Research and practice. Volume I: Learner char- 
acteristics and adaptive education (pp. 7-34). Oxford: Pergamon. 

Brophy, J. E. (1986). Research linking teacher behavior to student 
achievement: Potential implications for instruction of Chapter I stu- 
dents. In B. I. Williams, P. A. Richmond, & B. J. Mason (Eds.), De- 
signs for compensatory education: Conference proceedings and 
papers (pp. IV- 121 to IV- 179). Washington, DC: Research and Eval- 
uation Associates. 

Brophy, J. E., & Good, T. L. (1986). Teacher behavior and student 
achievement. In M. C. Wittrock (Ed.), Handbook of research on 
teaching (3rd ed., pp. 328-375). New York: Macmillan. 

Brown, B. W., & Saks, K. H. (1981). The microeconomics of schooling. 
In D. C. Berliner (Ed.), Review of research in education (Vol. 9, pp. 
217-254). Washington, DC: American Educational Research Associ- 
ation. 

Bryan, T. H., & Bryan, J. H. (1981). Some personal and social experi- 
ences of learning disabled children, in B. K. Keogh (Ed.), Advances in 
special education. Volume 3: Socialization influences on exceptional- 
ity (pp. 147-186). Greenwich, CT: JAI Press. 

Calfee, R. (1981). Cognitive psychology and educational practice. In 
D. C. Berliner (Ed.), Review of research in education (Vol. 9, pp. 
5-73). Was h i n g t on, DC: American Educational Research Association. 

Calfee, R. (1986). Curriculum and instruction: Reading. In B. I. Wil- 
liams, P. A. Richmond, & B. J. Mason (Eds.), Designs for compensa- 
tory education: Conference proceedings and papers (pp. IV-29 to 
IV -83). Washington, DC: Research and Evaluation Associates. 

Calfee, R., & Drum, P. (1986). Research on teaching reading. In M. C. 
Wittrock (Ed.), Handbook of research on teaching (23rd ed., pp. 
804-849). New York: Macmillan. 

Carpenter, R. L., & Apter, S. J. (1988). Research integration of cogni- 
tive interventions for behavio rally disordered children and youth. In 
M. C. Wang, M. C. Reynolds, & H. J. Walberg (Eds.), Handbook of 
special education. Volume 2: Mildly handicapped conditions (pp. 
155-159). Oxford: Pergamon. 

Carroll, J. B., & Maxwell, S. E. (1979). Individual differences in cogni- 
tive abilities. Annual Review of Psychology, 30, 603-640. 

Cazden, C. B. (1986). Classroom discourse. In M. C. Wittrock (Ed.), 
Handbook of research on teaching (3rd ed.'', pp. 432-463). New York: 
Macmillan. 

Clark, R., & Salomon, G. (1986). Media in teaching. In M. C. Wittrock 
(Ed.), Handbook of research on teaching (3rd ed., pp. 464-478). New 
York: Macmillan. 

Colarusso, R. P. (1987). Diagnostic-prescriptive teaching. In M. C. 
Wang, M. C. Reynolds, & H. J. Walberg (Eds.), Handbook of spe- 
cial education: Research and practice. Volume 1: Learner characteris- 
tics and adaptive education (pp. 155-166). Oxford: Pergamon. 

Cook, S. B., Scruggs, T. E., Mastropieri, M. A., & Gkndon, C. C. 
(1985-86). Handicapped students as tutors. Journal of Special Educa- 
tion, 19, 483-492. 

Cooper, H. M. (1986). Chapter 1 programs reduce student-to-instructor 
ratios but do reduced ratios affect achievement? In B. I. Williams, P. 

A. Richmond, & B. J. Mason (Eds.), Designs for compensatory 
education: Conference proceedings and papers (pp. 111-35 to 
111-57). Washington, DC: Research and Evaluation Associates. 

Como, L., & Snow, R. E. (1986). Adapting teaching to individual dif- 
ferences among learners. In M. C. Wittrock (Ed.), Handbook of re- 
search on teaching (3rd ed., pp. 605-629). New York: Macmillan. 

Cosden, M., Pear, R., Bryan, T. H. (1985). The effects of cooperative 
and individual goal structures on learning dreahfcri and noodisabled 
students. Exceptional Children, 52, 103-114. 

Craik, F. (1979). Human memory. Annual Review of Psychology, 30, 
63-102. 

Crittenden, K. S. (1983). Sodologica] aspects of attribution. Annual Re- 
view of Sociology, 9, 425-446. 

Curtis, M. E r , & Glaser, R. (1981). Changing c o nc e pt ions of intelli- 
gence. In D. C. Berliner (Ed.), Review of research in education (Vol. 
9, pp. 111-148). Washington, DC: American Educational Research 
Association. 

DeCharms, R., & Muir, M. S. (1978). Motivation: Social approaches. 
Annual Review of Psychology, 29, 91-1 14. 

de Kan ter, A., Ginsburg, A. L., & Milne, Am. M. (1986). Parent in- 
volvement strategies: A oew emphasis on traditional parent roles. In 

B. I. Williams, P. A. Richmond, & B. J. Mason (Eds.), Designs for 



compensatory education: Conference proceedings and papers (pp. 
V3-V26). Washington, DC: Research and Evaluation Associates. 

Diaz, R. (1983). Thought and two languages: The impact of bilingual- 
ism. In E. W. Gordon (Ed.), Review of research in education (Vol. 
10, pp. 23-54). Washington, DC: American Educational Research 
Association. 

Divesta, F., & Palermo, D. S. (1974). Language development. In F. N. 
Keriinger & J. B. Carroll (Eds.), Review of research in education 
(Vol. 2, pp. 55-107). Itasca, IL: Peacock. 

Doyle, W. (1977). Paradigms sfor research on teacher effectiveness. In 
L. S. Shulman (Ed.), Review of research in education (Vol. 5, pp. 
163-198). Itasca, IL: Peacock. 

Doyle, W. (1986a). Classroom organization and management. In M. C. 
Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 
392-431). New York: Macmillan. 

Doyle, W. (1986b). Vision and reality: A reaction to issues in curriculum 
and instruction for compensatory education. In B. 1. Williams, P. A. 
Richmond, & B. J. Mason (Eds.), Designs for compensatory educa- 
tion: Conference proceedings and papers (pp. IV -259 to IVn271). 
Washington, DC: Research and Evaluation Associates. 

Epps, S., & Tindel, G. (1987). The effectiveness of differential program- 
ming in serving students with mild handicaps: Placement options and 
instructional programming. In M. C. Wang, M. C. Reynolds, & H. J. 
Walberg (Eds.), Handbook of special education: Research and prac- 
tice. Volume 1: Learner ch ar ac t eri s tics and adaptive education (pp. 
213-247). Oxford: Pergamon. 

Epstein, M. H., Cullinan, D., & Rose, T. L. (1980). Applied behavior 
analysis and behavio rally disordered pupils: Selected issues. In L. 
Mann & D. A. Sabatino (Eds.), Fourth review of special education 
(pp. 61-107). New York: Grune & Stratton. 

Erickson, J. R., & Jones, M. R. (1978). Thinking. Annual Review of 
Psychology, 29, 61-90. 

Farr, M. (1986). Language culture and writing: Sock) linguistic founda- 
tions of reserch on writing. In E. Z. Rothkopf (Ed.), Review of re- 
search in education (Vol. 13, pp. 195-224). Washington, DC: Amer- 
ican Educational Research Association. 

Faust, M. S„ & Faust, W. *. (1980). Cognitive constructing: Levels of 
processing and developmental change. In B. K. Keogh (Ed.), Ad- 
vances in special education. Volume 1: Basic constructs and theoreti- 
cal orientations (pp. 1-54). Greenwich, CT: JAI Press. 

Fillmore, L., & Valdez, C. (1986). Teaching bilingual learners. In M. C. 
Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 
648-658). New York: Macmillan. 

Ford, B. G., & Jenkins, R. C. (1980). Changing perspectives in the edu- 
cation of the gifted. In L. Mann & D. A. Sabatino (Eds.), Fourth re- 
view of special education (pp. 151-174). New York: Grune & Stratton. 

Frase, L. (1975). Advances in research and theory in instructional tech- 
nology. In F. N. Keriinger (Ed.), Review of research in education 
(Vol. 3, pp. 43-74). Itasca. IL: Peacock. 

Gardner, M. (1985). Cognitive psychological approachs to instructional 
task analysis. In E. W. Gordon (Ed.), Review of research in educa- 
tion (Vol. 12, pp. 157-196). Washington, DC: American Educational 
Research Association. 

Gayne, R. (1973). In F. N. Keriinger (Ed.), Review of research in educa- 
tion (Vol. 1, pp. 5-33). Itasca, IL: Peacock. 

Gayne, R. M., & Dick, W. (1983). Instructional psychology. Annual 
Review of Psychology, 34, 261-296. 

Gecas, V. (1982). The seif-concept. Annual Review of Sociology, 8, 1-33. 

Gerber, M. M. (1987). Application of cognitive-behaviorist training 
methods to teaching baric skills to mildly handicapped elementary 
school students. In M. C. Wang, M. C. Reynolds, & H. J. Walberg 
(Eds.), Handbook of special education: Research and practice. Vol- 
ume 1: Learner ch ar a ct e ris tics and adaptive education (pp. 167-186). 
Oxford: Pergamon. 

Good, T., & Brophy, J. (1986). School effects. In M. C. Wittrock (Ed.), 
Handbook of research on teaching (3rd ed., pp. 570-654). New York: 
Macmillan. 

Gordon, E. W., DeStefano, L. M., & Shipman, S. (1985). Characteris- 
tics of learning persons and the adaptation of learning environments. 
In M. C. Wang & H. J. Walberg (Eds.), Adapting instruction to indi- 
vidual differences (pp. 44-65). Berkeley, CA: McCutchan. 

Gottlieb, B. W. (1987). Social facilitation effects in mainstream classes. 
In J. Gottlieb & B. W. Gottlieb (Eds.), Advances in special education 
(Vol. 6, pp. 75-85). Greenwich, CT: JAI Press. 






National Library of Education 



96 



Subject-Area Learning 



Journal of Educational Research 



Graves, M. F. (1986). Vocabulary learning and instruction. In E. Z. 
Rothkopf (Ed.), Review of r e s e arch in education (VoL 13, pp. 49-90). 
Washington, DC: American Educational Research A ssocia t ion. 

Greenspan. S. (1981a). Defining childhood social competence. In B. K. 
Keogh (Ed.), Advances in special education. Volume 3: Socialization 
influences on exceptionality (pp. 1-39). Greenwich, CT: JAI Press. 

Greenspan, S. (1981b). Social competence and handicapped individuals: 
Practical implications of a proposed model. In B. K. Keogh (Ed.), 
Advances in special education. Volume 3: Socialization influences on 
exceptionality (pp. 41-82). Greenwich, CT: JAI Press. 

Gresham, F. M. (1988). Social competence and motivational character- 
istics of lea rning HitahWi children. Jn M. C. Wang, M. C. Reynolds, 
& H. J. Walberg (Eds.), Handbook of special education. Volume 2: 
Mildly handicapped conditions (pp. 283-302). Oxford: Pergamon. 

Griffin, G. (1986). Chapter 1 and the regular school: Staff development. 
In B. I. Williams, P. A. Richmond, & B. J. Mason (Eds.), Designs 
for compensatory education: Conference proceedings and papers (pp. 
VI-41 to VI-58). Washington, DC: Research and Evaluation Associ- 
ates. 

Griswold, P. A., Colton, K., & Hansen, J. B. (1985). Effective compen- 
satory education sourcebook. Volume l-A review of effective educa- 
tional practices. Washington, DC U.S. Government Printing Office. 

Gump, P. V. (1980). The school as a social situation. Annual Review of 
Psychology, 31, 553-582. 

Hall, R. J. (1981). An information-processing approach to the study of 
exceptional children. In B. K. Keogh (Ed.), Advances in special edu- 
cation. Volume 2: Perspectives and applications (pp. 79-110). Green- 
wich, CT: JAI Press. 

HaUahan, D. P., & Reeve, R. E. (1980). Selective attention and distrect- 
ibility. In B. K. Keogh (Ed.), Advances in special education. Volume 
1: Basic constructs and theoretical orientations (pp. 141-181). Green- 
wich, CT: JAI Press. 

Hallman, M. T. (1086). Chapter 1 and student achievement: A concep- 
tual model. In B. 1. Williams, P. A. Richmond, & B. J. Mason 
(Eds.), Designs for compensatory education : Conference proceedings 
and papers (pp. VI-59 to VI-78). Washington, DC: Research and 
Evaluation Associates. 

Hamilton, S. F. (1984). The secondary school in the ecology of adoles- 
cent development. In E. W. Gordon (Ed.), Review of research in edu- 
cation (Vol. 11, pp. 227-258). Washington, DC: American Educa- 
tional Research Asxxaation. 

Hodges, W. L., & Buzzeli, C. A. (1984). School change and evaluation: 
Follow Through revisited. In B. K. Keogh (Ed.), Advances in special 
education. Volume 4: Documenting program impact (pp. 163-192). 
Greenwich, CT: JAI Press. 

Irvin, L. K., Close, D. W., & Wells, R. L. (1987). Programming inde- 
pendent living skills for handicapped learners. In M. C. Wang, M. C. 
Reynolds, & H. J. Walberg (Eds.), Handbook of special education: 
Research and practice. Volume I: Learner ch ar acteristics and adap- 
tive education (pp. 187-212). Oxford: Pergamon. 

Jackson, P. W. (1985). Private lessons in public schools: Remarks on 
the limits of adaptive instruction. In M. C. Wang & H. J. Walberg 
(Eds.), Adapting instruction to individual differences (pp. 66-81). 
B er kele y, CA: McClitchan. 

Johnson, D. W., & Johnson, R. T. (1985). Cooperative learning and 
adaptive education. In M. C. Wang & H. J. Walberg (Eds.), Adapt- 
ing instruction to individual differences (pp. 105-134). Berkeley, CA: 
McCutchan. 

Johnson, W. A., & Dark, V. J. (1986). Selective attention. Annual Re- 
view of Psychology, 37, 43-75. 

Karwcit, N. (1985). Tune spent, time needed, and adaptive instruction. 
In M. C. Wang & H. J. Walberg (Eds.), Adapting instruction to indi- 
vidual differences (pp. 281-297). Berkeley, CA: McCutchan. 

Kaufman, M., Agard, J. A., & Semmd, M. 1. (1985). Learner compe- 
tence in school. In M. Kaufman, J. A. Agard, & M. 1. Semmd, Main- 
streaming: Learners and their environment (pp. 99-150). Cambridge, 
MA: Brookline Books. 

Kaufman, M., Agard, J. A., & Semmd, M. 1. (1985). Teacher and peer 
characteristics. In M. Kaufman, J. A. Agard, and M. 1. Semmd, 
Mainstreaming: Learners and their environment (pp. 153-191). Cam- 
bridge, MA: Brookline Books. 

Kaufman, M., Agard, J. A., & Semmd, M. 1. (1985). The sodoemo- 
tipnai cl i mate . In M. Kaufman, J. A. Agard, and M. 1. Sen tin el, 
Mainstreaming: Learners and their environment (pp. 193-221). Cam- 



bridge, MA: Brookline Books. 

Kaufman, M., Agard, J. A., & Senund, M. 1. (1985). The instructional 
conditions. In M. Kaufman, J. A. Agard, and M. 1. Senund, Main- 
streaming: Learners and their environment (pp. 223-295). Cam- 
bridge, MA: Brookline Books. 

Kaufman, M., Agard, J. A., & Senund, M. 1. (1985). Academic compe- 
tence. In M. Kaufman, J. A. Agard, and M. 1. Senund, Mainstream- 
ing: Learners and their environment (pp. 307-364). Cambridge, MA: 
Brookline Books. 

Kaufman, M., Agard, A., & Senund, M. 1. (1985). Social compe- 
tence. In M. Kaufman, J. A. Agard, and M. 1. Senund, Mainstream- 
ing: Learners and their environment (pp. 365-400). Cambridge, MA: 
Brookline Books. 

Kavale, K. A. (1988). The long-term consequences of learning disabil- 
ities. In M. C. Wang, M. C. Reynolds, & H. J. Walberg (Eds.), 
Handbook of special education. Volume 2: Mildly handicapped con- 
ditions (pp. 303-344). Oxford: Pergamon. 

Kavale, K. A., & Glass, G. V. (1984). Meta-analysis and policy decisions 
in special education. In B. K. Keogh (Ed.), Advances in special educa- 
tion. Volume 4: Documenting program impact (pp. 195-247). Green- 
wich, CT: JAI Press. 

Kavale, K. A., & Nye, C. (1985-86). Parameters of learning disabilities 
in achievement, linguistic, neuropsychological, and social/behavioral 
domains. Journal of Special Education, 79(4), 443-458. 

Kdky, H. H., & Micheia, J. L. (1980). Attribution theory and research. 
Annual Review of Psychology, 31, 457-502. 

Kennedy, M. M., Jung, R. K., & Oriand, M. E. (1986). Poverty, 
achievement and the distribution of compensatory education services. 
In B. 1. Williams, P. A. Richmond, & B. J. Mason (Eds.), Designs 
for compensatory education: Conference proceedings and papers (pp. 
11-3 to 11-8). Washington, DC: Research and Evaluation Associates. 

Keogh, B. K. (1988). Learning disability: Diversity in search of order. In 
M. C. Wang, M. C. Reynolds, & H. J. Walberg (Eds.), Handbook of 
special education: Research and practice. Volume 2: Mildly handi- 
capped conditions (pp. 225-251). Oxford: Pergamon. 

Keogh, B. K., & Pullis, M. E. (1980). Temperament influences on the 
development of exceptional children. In B. K. Keogh (Ed.), Ad- 
vances in special education. Volume I: Bask constructs and theoreti- 
cal orientations (pp. 239-276). Greenwich, CT: JAI Press. 

Klausmeier, H. J. (1985). A design for improving secondary education. 
In M. C. Wang & H. J. Walberg (Eds.), Adapting instruction to indi- 
vidual differences (pp. 160-190). Berkeley, CA: McCutchan. 

Klausmeier, H. J., & Hooper, F. (1974). Conceptual development and 
instruction. In F. N. Keriinger & J. B. Carrol) (Eds.), Review of re- 
search in education (Vol. 2, pp. 3-54). Itasca, 1L: Peacock. 

Khiwin, T. N., & Moores, D. F. (1985). The effects of integration on the 
mathwTiatiet achievement of hearing-impaired adolescents. Excep- 
tional Children, 52, 153-160. 

Kramer, J. J., Piasd, W. C., & Glover, J. A. (1988). Cognitive and so- 
cial development of mildly retarded children. In M. C. Wang, M. C. 
Reynolds, & H. J. Walberg (Eds.), Handbook of special education: 
Research and practice. Volume 2: Mildly handicapped conditions 
(pp. 43-58). Oxford: Pergamon. 

Knipski, A. (1980). Attention proce sse s: Research, theory and implica- 
tions for special education. In B. K. Keogh (Ed.), Advances in special 
education. Volume 1: Bask constructs and theoretical orientations 
(pp. 101-140). Greenwich, CT: JAI Press. 

Langer, J. A;, & Applebee, A. N. (1986). Reading and writing instruc- 
tion: Toward a theory of teaching and learning. In E. Z. Rothkopf 
(Ed.), Review of research in education (Vol. 13, pp. 171-194. Wash- 
ington, DC: American Educational Research Association. 

Leyser, V., & Cole, K. B. (1987). The reconceptuaHzation and delivery 
of quality inservice education under Public Law 94-142. In J. Gott- 
lieb & B. W. Gottlieb (Eds.), Advances in special education (Vol. 6, 
pp. 87-117). Greenwich, CT: JAI Press. 

Liebert, R. M., & Schwartzberg, N. S. (1977). Effects of mass media. 
Annual Review of Psychology, 28, 141-174. 

Lloyd, J. W. (1988). Direct academic interventions in learning disabil- 
ities. In M. C. Wang, M. C. Reynolds, & H. J. Walberg (Eds.), 
Handbook of special education. Volume 2: Mildly handicapped con- 
ditions (pp. 345-366). Oxford: Pergamon. 

Lloyd, J. W., HaJHahan, D. P., & Kaufman, J. M. (1980): Learning dis- 
abilities: A review of elected topics. In L. Mann & D. A. Sabatino 
(Eds.), Fourth review of special education (pp. 35-60). New York: 




97 



Volume 2, Spring 1997 



Subject-Area Learning 



September/October 1990 | Vol. 84<No. 1)J 



Grune Sl Stratton. 

Macdonald* Ross, M. (1978). Language in texts: The design of curricular 
materials. In L. S. Shulman (Ed.), Review of research in education 
(Vol. 6, pp. 229-273). Itasca, IL: Peacock. 

MacMillan, D. L., Keogh, B. K., St Jones, R. L. (1986). In M. C. Wht- 
rock (Ed.), Handbook of research on teaching (3rd ed., pp. 686-724). 
New York: Macmillan. 

MacMillan, D. L., St Meyers, C. E. (1979). Educational labeling of 
handicapped learners. In D. C. Berliner (Ed.), Review of research in 
education (Vol. 7, pp. 131-194). Washington, DC: American Educa- 
tional Research Association. 

Mahoney, G., Crawley, S., St Pullis, M. ( 1980). Language intervention: 
Models and issues. In B. K. Keogh (Ed.), Advances in special educa- 
tion. Volume 2: Perspectives and applications (pp. 133-164). Green- 
wich, CT: JA1 Press. 

Mason, J. M., & Allen, J. (1986). A review of emer gen t literacy with im- 
plications for research and practice in reading. In E. Z. Rothkopf 
(Ed.), Review of research in education (Vol. 13, pp. 3-47). Washing, 
ton, DC: American Educational Research Association. 

McConkie, G. W. (1977). Learning from test. In L. S. Shulman (Ed.), 
Review of research in education (Vol. 3, pp. 3-48). Itasca, IL: 
Peacock. 

McLaughlin, M. W., St Shields, P. M. (1986). Involving parents in the 
school: Lessons for policy. In B. I. Williams, P. A. Richmond, St B. 
J. Mason (Eds.), Designs for compensatory education: Conference 
proceedings and papers (pp. V-27 to V-46). Washington, DC: Re- 
search and Evaluation Associates. 

Meisgeier, C. (1976). A review of critical issues underlying mainstream- 
ing. In L. Mann & D. A. Sabatino (Eds.), Third review of special edu- 
cation (pp. 245-269). New York: Grune St Stratton. 

Merrill, M. D., & Boutwell, R. C. (1973). Instructional development: 
Methodology and research. In 'F. N. Kerlingr (Ed.), Review of re- 
search in education (Vol. 1, pp. 95-131). Itasca, IL: Peacock. 

Morrison, G. M. (1987). Relationship among academic, and 

career education in programming for handicapped students. In M. C. 
Wang, M. C. Reynolds, St H. J. Walberg (Eds.), Handbook of spe- 
cial education: Research and practice. Volume 1: Learner characteris- 
tics and adaptive education (pp. 133-154). Oxford: Pergamon. 

Morsink, C. V., Thomas, C. C., St Smith-David, J. (1987). Noncaiegor- 
ical special education programs: Process and outcomes. In M. C. 
Wang, M. C. Reynolds, St H. J. Walberg (Eds.), Handbook of spe- 
cial education: Research and practice. Volume I: Learner characteris- 
tics and adaptive education (pp. 287-31 1). Oxford: Pergamon. 

Nelson, C. M., St Rutherford, R. B. (1988). Behavioral interventions 
with behaviorally disordered students. In M. C. Wang, M. C. Rey- 
nolds, St H. J. Walberg (Eds.), Handbook of special education: Re- 
search and practice. Volume 2: Mildly handicapped conditions (pp. 
125-153). Oxford: Pergamon. 

Nelson-LeGall, S. (1985). Help-seeking behavior in learning. In E. W. 
Gordon (Ed.), Review of research in education (Vol. 12, pp. 55-90). 
Washington, DC: American Educational Research Association. 

Ncvin, A., St Thousand, J. (1987). Avoiding or limiting special educa- 
tion referrals: Changes and Challenges. In M. C. Wang, M. C. Rey- 
nolds, St H. J. Walberg (Eds.), Handbook of special education: Re- 
search and practice. Volume I: Learner ch ar ac t eristics and adaptive 
education (pp. 273-286). Oxford: Pergamon. 

Niemiec, R., St Walberg, H. J. (1987). Comparative effects of com- 
puter-assisted instruction: A synthesis of reviews, Journal of Educa- 
tional Computing Research, 3, 19-37. 

Olsen, K. R. (1986). Effectiveness indicators for special education. 
Hampton, NH: Center for Resource Management. (Developed by the 
National RRC Panel on Indicators of Effectiveness in Special Educa- 
tion, K. R. Olsen, chair.) 

Palincsar, A. S., & Brown, A. (1987). Advances in improving the cogni- 
tive performance of handicapped students. In M. C. Wang, M. C. 
Reynolds, St H. J. Walberg (Eds.), Handbook of special education: 
Research and practice. Volume l: Learner characteristics and adap- 
tive education (pp. 93-112). Oxford: Pergamon. 

Paris, S. G., Wixson, K. K., St Palincsar, A. S. (1986). Instructional ap- 
proaches to reading comprehension. In E. Z. Rothkopf (Ed.), Review 
of research in education (Vol. 13, pp. 91-128). Washington, DC: 
American Educational Research Association. 

Passow, A. H. (1974). Compensatory instructional intervention. In F. 
N. Kerlinger and J. B. Carroll (Eds.), Review of research in education 



(Vol. 2, pp. 145-175). Itasca, IL: Peacock. 

Pea, R. D., St Kurland, D. M. (1987). Cognitive technologies for writ- 
ing. In E. Z. Rothkopf (Ed.), Review of research in education Vol. 
14, pp. 277-326). Washington, DC: American Educational Research 
Association. 

Peterson, P. L. (1986). Selecting students and services for compensatory 
education: Lessons from aptitude-treatment interaction research. In 
B. I. Williams, P. A. Richmond, St B. J. Mason (Eds.), Designs for 
compensatory education: Conference proceedings and papers (pp. 
11-15 to 11-62). Washington, DC: Research and Evaluation 
Associates. 

Pintrich, P. R., Cross, D. R., Kozina, R. B., St McKeachie, W. J. 
(1986). Instructional psychology. Annual Review of Psychology, 37, 
611-651. 

Polirstok, S. R. (1987). A specialized peer tutoring program for aca- 
demically and behaviorally handicapped adolescents. In J. Gottlieb St 
B. W. Gottlieb (Eds.), Advances in special education (Vol. 6, pp. 
63-74). Greenwich, CT: JAI Press. 

Polioway, E. A., St Smith, J. D. (1988). Current status of the mild men- 
tal retardation construct: Identification, placement, and programs. In 
M. C. Wang, M. C. Reynolds, St H. J. Walberg (Eds.). Handbook of 
special education: Research and practice. Volume 2: Mildly handi- 
capped conditions (pp. 7-22). Oxford: Pergamon. 

Price- Williams, D., St Gallimore, R. (1980). The cultural per sp ective. In 
B. K. Keogh (Ed.), Advances in special education. Volume 2: Per- 
spectives and applications (pp. 165-192). Greenwich, CT: JAI Press. 

Ramey, C. T., Yeates, K. O., St MacPhee, D. (1984). Risk for retarded 
development among disadvantaged families: A systems theory ap- 
proach to preventive intervention. In B. K. Keogh (Ed.), Advances in 
special education. Volume 4: Documenting program impact (pp. 
249-272). Greenwich, CT: JAI Press. 

Reschly, D. J. (1987). Learning characteristics of mildly handicapped 
students: Implications for classification, placement and program- 
ming. In M. C. Wang, M. C. Reynolds, St H. J. Walberg (Eds.), 
Handbook of special education: Research and practice. Volume l: 
Learner characteristics and adaptive education (pp. 35-58). Oxford: 
Pergamon. 

Resnick, L. B. (1981). Instructional psychology. Annual Review of Psy- 
chology, 32, 659-704. 

Reynolds, M. C. (1984). Classification of students with handicaps. In E. 
W. Gordon (Ed.), Review of research in education (Vol. 11, pp. 63-92). 
Washington, DC: American Educational Research Association. 

Reynolds, M. C., St Lakin, K. C. (1987). Noncat egorical special educa- 
tion: Models for research and practice. In M. C. Wang, M. C. Rey- 
nolds, St H. J. Walberg (Eds.), Handbook of special education: Re- 
search and practice. Volume l: Learner cha r ac t er is tics and adaptive 
education (pp. 313-329). Oxford: Pergamon. 

Rich, D. (1986). The parent gap in compensatory education and how to 
bridge it. In B. J. Williams, P. A. Richmond, St B. J. Mason (Eds.), 
Designs for compensatory education: Conference proceeding? and 
papers (pp. V-47 to V-62). Washington, DC: Research Evaluation 
Associates. 

Romberg, T. (1986). Mathematics for compensatory school programs. 
In B. I. Williams, P. A. Richmond, St B. J. Mason (Eds.), Designs 
for compensatory education: Conference proceedings and papers (pp. 
IV-3 to IV-28). Washington, DC: Research and Evaluation 
Associates. 

Romberg, T. A., St Carpenter, T. P. (1986). Research on teaching and 
learning mathematics. Two disciplines of scientific inquiry.In M. C. 
Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 
850-873). New York: Macmillan. 

Rosenshine, B., St Stevens, R. Teaching functions. (1986). In M. C. 
Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 
376-391). New York: Macmillan. 

Ruble, D. N„ St Boggiano, A. K. (1980). Optimizing motivation in an 
achievement context. In B. K. Keogh (Ed.), Advances in special edu- 
cation. Volume 2: Perspectives and applications (pp. 183-238). 
Greenwich, CT: JAI Press. 

Sarason, S. B., St Klaber, M. (1985). The school as a social situation. 
Annual Review of Psychology, 36, 1 15-140. 

Scardamaha, M., St Berciter, C. (1986). Research on written composi- 
tion. In M. C. Wittrock (Ed.), Handbook of research on teaching 
(3rd ed., pp. 778-803). New York: Macmillan. 

Schmidt, M., Weinstein, T., Niemiec, R., St Walberg, H. J. (1985-86). 



o 

ERIC 



National Library of Education 



98 



Subject-Area Learning 



Computer-assisted instruction with exceptional children. Journal of 
Special Education 19(A), 493-501. 

Scott-Jones, D. (1974). Family influences on cognitive development and 
school achievement. In E. W. Gordon (Ed.), Review of research in 
education (Vol. 11, pp. 259-304). Washington, DC: American Edu- 
cational Research Asso ciat ion. 

Semmd, M. 1., & Frick, T. (1985). Learner background characteristics. 
In M. Kaufman, J. A. Agard, & M. 1. Semmel, Mainstreaming: 
Learners and their environment (pp. 75-98). Cambridge, MA: Brook- 
line Books. 

Semmel, M. 1., Gottlieb, J., & Robinson, N. M. (1979). Mainstreaming: 
Perspectives on educating handicapped children in the public school. 
In D. C. Berliner (Ed.), Review of research in education (Vol. 7, pp. 
223-279). Washington, DC: American Educational Research 
Association. 

Shipman, S.. & Shipman, V. C. (1985). Cognitive styles: Some concep- 
tual, methodological, and applied issues. In E. W. Gordon (Ed.), 
Review of research in education (Vol. 12, pp. 229-291). Washington, 
DC: American Educational Research Association. 

Shores, R. E., Bumey, J. D., & Wiegerink, R. (1976). Teacher training 
in special education: A review of research. In L. Mann & D. A. Saba- 
tino (Eds.), Third review of special education (pp. 199-216). New 
York: Grune & Stratton. 

Skiba, R. J., Casey, A., & Center, B. A. (1985-86). Nonaversive proce- 
dures in the treatment of classroom behavior problems. Journal of 
Speckd Education 19(A), 459-481. 

Slavin, R. E. (1985). Team-assisted indidivudlizaiion: A cooperative 
learning solution for adaptive instruction in mathematics. In M. C. 
Wang & H. J. Walberg (Eds.), Adapting instruction to individual dif- 
ferences (pp. 236-253). Berkeley, CA: McCutchan. 

Smith, C. R., Wood, F. H., & Grimes, J. (1988). Issues in the identifica- 
tion and placement of behavioraHy disordered students. In M. C. 
Wang, M. C. Reynolds, & H. J. Walberg (Eds.), Handbook of spe- 
cial education: Research and practice. Volume 2: Mildly handicapped 
conditions (pp. 95-123). Oxford: Pergamon. 

Snow, R. E. (1976). Learning and individual differences. In L. S. Shul- 
man (Ed.), Review of research in education (Vol. 4, pp. 50-105). 
Itasca, IL: Peacock. 

Spady, W. G. (1973). The impact of school resources on students. In F. 
N. Kertinger (Ed.), Review of research in education (Vol. 1, pp. 
135-177). Itasca, 1L: Peacock. 

Stainback, W., Stain back, S., Court nage, L., & Jaben, T. (1985). Fa- 
cilitating mainstreaming by modifying the mainstream. Exceptional 
Children, 52, 144-152. 

Stei n, N. L. (1986). Knowledge and process in the acquisition of writing 
skills. In E. Z. Rothkopf (Ed.), Review of research in education (Vol. 
13, pp. 225-258). Washington, DC: American Educational Research 
Association. 

Taylor, R. L. (1988). Psychological intervention with mildly retarded 
children: Prevention and remediation of cognitive deficits. In M. C. 
Wang, M. C. Reynolds, & H. J. Walberg (Eds.), Handbook of spe- 
cial education: Research and practice. Volume 2: Mildly handicapped 
conditions (pp. 59-75). Oxford: Pergamon. 

Tobias, S. (1985). Computer-assisted instruction. In M. C. Wang & H. 
J. Walberg (Eds.), Adapting instruction to individual differences (pp. 
135-159). Berkeley, CA: McCutchan. 

Torgesen, J., & Kail R. V. (1980). Memory processes in exceptional 
children. In B. K. Keogh (Ed.), Advances in special education. 
Volume I: Bade constructs and theoretical orientations s (pp. 55-99). 
Greenwich, CT: JA1 Press. 

Torrance, P. (1986). Teaching creative and gifted learners. In M. C. 
Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 
630-647). New York: Macmillan. 

Tyler, W. B. (1985). The organizational structure of the school. Annual 
Review of Sociology, II, 49-74. 

U.S. Department of Education. (1986). What works. Washington, DC: 
U.S. Government Printing Office. 

Wachs, H., & Furth, H. (1980). Piaget’s theory and special education. 
In B. K. Keogh (Ed.), Advances in special education. Volume 2: Per- 
spectives and applications (pp. 51-78). Greenwich, CT: JA1 Press. 

Walberg, H. J. (1976). Psychology of learning environments: Behav- 
ioral, structural or p erce p tual? in L. S. Shulman (Ed.), Review of re- 
search in education (Vol. 4, pp. 142-178). Itasca, 1L: Peacock. 

Walberg, H. J. (1984). Improving the productivity of America’s 



Journal of Educational Research 



schools. Educational Leadership, 47(8), 19-27. 

Walberg, H. J. (1985). Instructional theories and research evidence. In 
M. C. Wang & H. J. Walberg (Eds.), Adapting instruction to individ- 
ual differences (pp. 3-23). Berkeley, CA: McCutchan. 

Walberg, H. J. (1986). Syntheses of research on teaching. In M. C. 
Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 
214-229). New York: Macmillan. 

Walberg, H. J., & Wang, M. C. (1987). Effective educational practices 
and provisions for individual differences. In M. C. Wang, M. C. Rey- 
nolds, & H. J. Walberg (Eds.), Handbook of special education: Re- 
search and practice. Volume I: Learner characteristics and adaptive 
education (pp. 113-128). Oxford: Pergamon. 

Wallace, R. C., Jr. (1985). Adaptive education: Policy and administra- 
tive perspect i ves. In M. C. Wang & H. J. Walberg (Eds.), Adapting 
instruction to individual differences (pp. 269-280). Berkeley, CA: 
McCutchan. 

Wang, M. C., & Baker, E. T. (1985-86). Mainstreaming programs: De- 
sign features and effects. Journal of Special Education, 19, 503-521. 

Wang, M. C., Gennari, P., & Waxman, H. C. (1985). The adaptive 
learning environments model: Design, implementation, and effects. 
In M. C. Wang & H. J. Walberg (Eds.), Adapting instruction to indi- 
vidual differences (pp. 191-235). Berkeley, CA: McCutchan. 

Wang, M. C., & Lindvall, C. M. (1984). Individual differences and 
school learning environments. In E. W. Gordon (Ed.), Review of re- 
search in education (Vol. 11, pp. 161-225). Washington, DC: Amer- 
ican Educational Research Association. 

Wang, M. C., & Peverty. S. T. (1987). The role of the learner: An indi- 
vidual difference variable in schol learning and functioning. In M. C. 
Wang. M. C. Reynolds, & H. J. Walberg (Eds.), Handbook of 
special education : Research and practice. Volume 1: Learner 
characteristics and adaptive education (pp. 59-92). Oxford: 
Pergamon. 

Wang, M. C., Peverty, S. T., & Catalano, R. (1987). Integrating special 
needs students in regular classes: Programming, implementation, and 
policy. In J. Gottlieb 8c B. W. Gottlieb (Eds.), Advances in special 
education (Vol. 6, pp. 119-149). Greenwich, CT: JA1 Press. 

Waxman, H. C., &. Walberg, H. J. (1982). The relation of teaching and 
learning: A review of reviews of process-product research. Contem- 
porary Education Review, I, 103-120. 

Weiner, B. (1976). An attributional approach to education psychology. 
In L. S. Shulman (Ed.), Review of research in education (Vol. 4, pp. 
179-209). Itasca, 1L: Peacock. 

Weinstein, C. F., & Mayer, R. F. (1986). The teaching of learning strat- 
egies. In M. C. Wittrock (Ed.), Handbook of research on teaching 
(3rd ed., pp. 315-327). New York: Macmillan. 

White, R. T., & Tisher, R. P. (1986). Research on natural sciences. In 
C. Wittrock (Ed.), Handbook of research on teaching (3rd ed., pp. 
874-905). New York: Macmillan. 

Wickdgren, W. A. (1981). Human learning and memory. Annual Re- 
view of Psychology, 32, 7-1-52. 

Wilkinson, L. C. (1986). Grouping low achieving students for instruc- 
tion. In B. 1. Williams, P. A. Richmond, & B. J. Mason (Eds.), De- 
signs for compensatory education: Conference proceedings and 
papers (pp. FV-181 to 1V-206). Washington, DC: Research and Eval- 
uation Associates. 

Wittrock, M. C. (1986). Student thought processes. In M. C. Wittrock 
(Ed.), Handbook of research on teaching (3rd ed., pp. 297-314). New 
York: Macmillan. 

Wittrock, M. C., & Lumsdaine, A. A. (1977). Instructional psychology. 
Annual Review of Psychology, 28, 417-460. 

Zabel, R. H. (1988). Preparation of teachers for behavioraHy distorted 
students: A review of the literature. In M. C. Wang, M. C. Reynolds, 
&H. J. Walberg (Eds.), Handbook of special education: Research 
and practice. Volume 2* Mildly handicapped conditions (pp. 
171-193). Oxford: Pergamon. 

Zigmond, N., & Sainatom, D. (1981). Socialization influences on educa- 
tionally handicapped adolescents. In B. K. Keogh (Ed.), Advances in 
speckd education. Volume 3: Socialization influences on exceptional- 
ity (pp. 187-207). Greenwich, CT: JA1 Press. 

REFERENCES 

Bennett, S. N. (1978). Recent research on teaching: A dream, a belief 
and a model. British Journal of Educational Psychology, 48, 




103 

99 Volume 2, Spring 1997 



Subject-Area Learning 



Scptember/October 1990 ( Vol. 84(No. I)} 



127*147. 

Bloom, B. S. (1976). Human Characteristics and School Learning New 
York: McGraw-Hill. 

Brookova. W. B.. Body, C.. Hood. P.. Sch J„ & wisenboker, 
J. (1979). School social systems and student achievement: Schools am 
make a difference. New York: Pracger. 

Brook over, W. B., A Lerotte, L. W. (|979). Changes in school charac- 
teristics coincident with changes in student achievement. East Lans- 
ing, Ml: Michigan State University. Institute for Research on Teach- 
ing. (ERJC Document Reproduction Service No. ED 181 005) 

Brophy, J. (1986). Research linking teacher behavior to student achieve- 
ment: Potential implications for instruction of Chapter I students. In 
B. 1. Williams, P. A. Richmond, A B. J. Mason (Eds.). Designs for 
compensatory education : Conference proceedings and papers. Wash- 
ington. DC: Research and Evaluation Associates. Inc. 

Bruner, J. S. (1966). Toward a theory of instruction. New York: W. W. 
Norton. 

Canoll. J. B. (1963). A model of school learning. Teachers College Rec- 
ord. 64. 723-733. 

Coleman. J. S. et al. (1966). Equality of educational opportunity. 
Washington. DC: U_S. Department of Health. Education, and Wel- 
fare. Office of Education. 

Edmonds. R. (1979). Effective schools for the urban poor. Educational 
Leadership . JZ(I). 15-27. 

Glaser. R. (1976). Components of a psychological theory of instruction: 
Toward a science of design- Review of Educational Research 46 
1-24. 

Glaser. R. (1982). Instructional psychology: Past, present, and future. 

Amencan Psychologist. 37. 292-305. 

Haertel. G. D.. Walbcrg. H. J., A Weinstein. T. (1983). Psychological 
models of educational performance: A tlieorcticaJ synthesis of con 
structs. Review of Educational Research. 33. 75-92. 



Hamischfeger. A., A Wiley. D. E. (1976). The teaching-learning process 
in elementary schools: A synoptic view. Curriculum Inquiry, 6. 5-43. 
Hilgard, E. R. (1964). A perspective of the relationship between learn- 
ing theory and educational p racti c es . In E. R. Hilgard (Ed.). Theories 
of teaming and instruction (Sixty-third yearbook of the National 
Society for the Study of Education, pp. 402-415). Chicago: Univer- 
sity of Chicago Press. 

Keogh. B. K.. Major- Kingsley, S., Omori -Gordon, H., A Reid. H. P. 
(1982). A system of marker variables for the field of teaming disabil- 
ities. Syracuse. NY: Syracuse University Press. 

Purkcy, S. C.. A Smith. M. S. (1983). Effective schools: A review. Ele- 
mentary School Journal. 83. 427-452. 

Rutter. M., Maugham. B.. Monimore, P., Oust on. J.. A Smith. A. 
(1979). Fifteen thousand hours: Secondary schools and their effects 
on children. Cambridge, MA: Harvard University Press. 

Shulman. L. S. (1986). Paradigms and research programs in the studv 
of teaching. In M. C. Wittrock (Ed.). Handbook of research on 
teaching (3rd cd.. pp. 3-36). New York: Macmillan. 

U.S. Depart mem of Education. (1986). What works: Research about 
teaching and teaming. Washington. DC: Author. 

Wang. M. C., A Lind vail. C. M. (1984). Individual differences and 
school learning environment. In E. W. Gordon (Ed.). Review of 
Research in Education (pp. 161-225). Washington, DC: American 
Educational Research Association. 

Wang, M. C-. A Walbcrg, H. J. (1985). Adapting instruction to individ- 
ual differences. Berkeley. CA: McCutchan. 

Wiliams. B. I.. Richmond. P. A.. A Mason. I), j. (Eds.). (1986). De- 
signs for compensatory education: Conference proceedings and pa- 
pers. Washington. DC: Research and Evaluation Associates. Inc. 
Wittrock. M. C- (Ed.). (1986). Handbook of research on teaching (3rd 
ed.). A project of the American Educational Research Association. 
New York: Macmillan. 



O 

ERiC 



National Library of Education 



100 



Subject-Area Learning 



TEMPLE UNIVERSITY CENTER FOR RESEARCH 
IN HUMAN DEVELOPMENT AND EDUCATION 



The Temple University Center for Research in Human Development and Education is an interdisciplinary center for 
the study of emerging problems and challenges facing children, youth, and families. Its overall goal is to investigate the basic 
forces that affect human development as well as educational proccsscsand outcomes. An important focus of the Center's work 
is the identification and shaping of effective responses to these forces through far-reaching changes in institutional policies and 
practices. 



The problems and challenges facing children, youth, and families stem from a variety of cultural, economic, political, 
and health pressures. Their solutions arc. by nature, complex. They require long-term programs of study that apply knowledge 
and expertise from many disciplines and professions. To this end. the Center draws together the many resources of the 
University and a wide range of national, state, and regional programs. The result is interdisciplinary and interdepartmental 
collaborations that involve investigations of social, economic, educational, and developmental factors and demonstration of slafo- 
of-thoart models for training and for delivery of relevant services. Research and development projects in these areas reflect 
a commitment to enhance the knowledge base for improving the quality of life for children and families, particularly in urban 
environments. 

The work of the Center for Research in Human Development and Education is divided into four program units. 
Improving Instruction and Learning in Schools, which provides technical assistance and training for innovative school programs; 
Social Service Delivery Systems, which develops models for effective social service delivery; Studies of Child Development 
and Early Intervention, which conducts pre-school diagnosis and produces innovative program development; and the National 
Center on Education in’ the Inner Chics (CEIC). funded by the US. Department of Education's Office of Educational Research 
and Improvement, which has undertaken a program of research and development as well as dissemination that lakes bold steps 
to mobilize and strengthen education and related resources to foster resilience and learning success of children, youth, and their 
families in inner chics. 



The Center is supported by funds from Temple University as well as by grants from public agencies and private 
foundations. The following arc the current funding sources: 



Bell Atlantic 

Ben Franklin Partnership Program of the Advanced 
Technology Center for S. E. Pennsylvania 
Carnegie Corporation 
City of Philadelphia 

Department of Human Services 
Mayor's Commission on Literacy 
Office of Mental Health and Mental Retardation 
School District of Philadelphia 
Commonwealth of Pennsylvania 

Department of Education 

Bureau of Special Education 
Bureau of Vocational and Adult Education 
Department of Labor and Industry 
Department of Public Welfare 
Delaware State Department of Education 



Exxon Education Foundation 
Geraldine R. Dodge Foundation 
IBM Corporation 

Louisiana State Department of Education 
National Science Foundation 
Private Industry Council of Pltila. 

Rockefeller Foundation 

The Pew Charitable Trusts 

United States Department of Education 

Office of Educational Research &. Improvement 
Office of Efcmentary Sc Secondary Education 
Office of Special Education Programs 
United States DcpC of Health Sc Human Services 
United States Department of Labor 
William Penn Foundation 
William T. Grant Foundation 



For further information, please contact: 

Temple University Center for Research 
in Human Development and Education 
13th Sc Cecil B. Moore Avenues 
Philadelphia, PA 19122 
(215) 787-3000 



105 



101 



Volume 2, Spring 1997 



Subject- Area Learning 

* International Reading Association 703 1 -4620/9 1 /USS 1 .25 + .00 



Isabel L. Beck 
Margaret G. McKeown 
Gale M. Sinatra 
Jane A. Loxterman 
University of Pittsburgh 



Revising social studies text from a 
text-processing perspective: Evidence of 
improved comprehensibility 



the purpose of the present study was to use a cognitive processing perspective to revise fifth- 
grade social studies texts, to describe those revisions, and to demonstrate their effects empiri- 
cally. Four segments of text from a U.S. textbook about the period leading to the American 
Revolution and their revised counterparts were presented to 85 fourth- and fifth-grade stu- 
dents. Students were presented with the text materials in individual sessions with an examiner 
and were asked to recall what they had read and to answer questions on the material. Students 
who read the revised text recalled more material and answered more questions correctly than 
students who read the original text. Differences in understanding between the two groups 
were also captured in qualitative analyses of recall protocols and question responses related to 
specific ideas in the text. Overall, the effects of the revisions demonstrate that a text-process- 
ing approach to creating comprehensible text is a viable one. Furthermore, the description of 
revision goes beyond how revisions have been described in past research by exposing the 
reasoning underlying the identification of problems and the changes made. 

Amelioration de la comprehensibilite des textes en sciences humaines: 

Elaboration de criteres de revision fondes sur les modeles cognitifs de 
traitement des textes 

le but de la presente recherche etait d'appliquer la perspective des modules cognitifs de 
traitement des textes k la revision de textes extraits de manuels scolaires en sciences humaines 
du niveau de cinquifeme annee primaire. La recherche a porte sur quatre extraits dun livre 
dliistoire traitant de la p^riode qui a conduit k la revolution am€ricaitfe. Les versions 
originales des quatre textes et les versions revisees ont €l€ presentees a 85 eifcvcs de quatri&roe 
et de cinquifcme annee au cours de sessions individuelles; les sujets devaient produire un 
rappel des textes et repondre k des questions de comprehension. Les eifcves qui ont lu les 
versions revisees ont rappeie davantage d'informations et ont repondu a davantage de 
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questions que les sujets qui ont lu !cs versions originalcs. Dcs analyses qualitativcs 
appnqudcs a la fois sur les rappels ct sur les rdponscs aux questions dc comprehension ont 
permis dc mieux saisir les diffdrcnccs de comprehension. En gendral. les rcsultats ont 
confirmd I intdret dappliquer un moddlc dc traitement cognitif dcs textes pour ameliorcr la 
comprehensibilitd des textes. La prdsente recherche decrit explicitement le rationcl qui 
motive les amdliorations apportdes, la fa?on d'identificr les probldmcs ct la nature dcs 
changcmcnts apportds. 

Revisando un texto de estudios sociales desde una perspectiva de 
procesamiento de textos: Evidencia de mejora en la comprension 

el prop6sito del presente cstudio fuc usar una perspectiva del procesamiento cognitivo para 
revisar los textos dc quinto grado dc estudios sociales, dcscribir esas rcvisioncs. v demostrar 
sus cfectos cmpiricamentc. Cuatro segmentos dc textos dc un libro dc texto ace rca del 
pcriodo anterior a la Rcvolucidn norteamericana fucron revisados. Sc presentaron los 
segmentos originalcs y las versioncs revisadas a 85 estudiantes dc cuarto y quinto grados en 
scsioncs individuals; sc pidid a los estudiantes que rccordaran lo que habfan letdo y que 
contestaran preguntas sobre cl material. Los estudiantes que leycron el texto revisado 
rccordaron mis material y contcstaron mas preguntas corrcctamcnte que los estudiantes que 
cycron cl texto original. Tambien se idcntificaron difcrcncias en la comprension en los dos 
grupos en andlisis cualitativos dc protocolos de rccucrdo y respuestas a preguntas 
relacionadas con ideas espec.'ficas en el texto. En general, los cfectos dc las revisiones 
demostraron que un accrcamiento dc procesamiento dc texto para crcar un texto comprcnsivo 
cs una aproximacion viable. La investigacidn actual describe cxplfcitamcntc la razon para 
cada revisidn hccha-como sc identified cada problcma. y que cambios fucron hcchos para 

enfrenfnr nrnn nmr» r 



Landeskundliche Texte aus Sicht einer textverarbeitenden Perspektive revi- 
dieren: Belegefiir eine verbesserte Verstandlichkeit des Textes 

zweck dieser studie war cs. Forschungscrgcbnissc im Bcrcich dcr kognitiven Vcrurbcitung 
anzuwenden. urn landeskundliche Tcxtc dcs fiinftcn Schuljahrcs zu rcvidicrcn. diese 
Bcarbeitungen zu beschrcibcn und deren Wirkungcn cmpirisch nachzuwciscn. Aus cincm 
amcrikamschcn Lchrbuch wurden dazu vicr Tcxtabschnittc, die die Zcit vor dcr 
Amerikanischen Revolution bchandeltcn, revidiert. Die Originalabschnittc und die 
rcvidierten Fassungen wurden 85 Schiilcm des 4. und 5. Schuljahrcs vorgclcgt. Bcidc Male 
wurden die Abschnitte im Einzcluntcrricht prasentiert und die Schuler dazu aufgcfordcrt. das 
Gclesenc nachzucrzahlen und Fragcn zum Material zu beantworten. Die Schuler, die den 
revidiertcn Text gelcsen hatten. konnten mchr Informationcn nachcrzahlcn und mchr Fraccn 
nehtig beantworten als die Schuler, die den Originaltcxt gelcsen hatten. Die Untcrschicdc im 
Vcrstandms zwischen den beiden Gruppcn wurden auflerdem anhand von qualitativcn 
Auswertungcn der Nachcrzahlungcn und Antwortcn auf die Tcstfragcn. die sich auf 
bestimmte Ideen des Textes bezogen. fcstgchaltcn. Im ganzen zeigen die Auswirkunccn dcr 
Revisionen. dal) cin textverarbeitendes Vcrfahren bcim Aufstcllcn von Vcrstandni^stcxtcn 
smnvoll ,st. In dcr vorliegenden Studie werden die Grundprinzipicn jeder cinzclncn 
Revidierung besprochen: wie jcdcs einzelnc Problem identifiziert wurdc und wclchc 
Andcrungen zur Behcbung dcs jeweiligen Problems in Angriff genomnicn wurden. 
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T he quality of school textbooks has long 
been of concern to professionals in many 
areas of educational research and practice. One 
way this concern has been expressed is in at- 
tempts to provide students with “comprehen- 
sible” texts, which have traditionally been 
manifested through the application of readabil- 
ity formulas. These formulas are based on sim- 
ple indices, such as sentence and word length, 
that correlate with text difficulty, but do not ac- 
count for what makes a text easier or more diffi- 
cult to comprehend (Kintsch & Vipond, 1979). 

Recently, cognitive theorists and reading 
researchers have gone beyond the conception of 
comprehensibility as simply vocabulary diffi- 
culty and sentence length, and have begun to 
describe text features that influence compre- 
hension directly, by considering how readers 
construct a cognitive representation of incom- 
ing information. Such a cognitive perspective, 
with its emphasis on the mental activities in- 
volved in reading, has yielded an understanding 
of reading as a complex process in which a 
reader constructs meaning by integrating per- 
ceptual, linguistic, and conceptual information 
from the text with his or her own knowledge 
base (see, e.g., Graesser & Clark, 1985; Just & 
Carpenter, 1987; Kieras, 1985; Perfetti, 1985; 
Rumclhart, 1980; Trabasso, Secco, & van den 
Broek, 1984; van Dijk & Kintsch, 1983). Cog- 
nitive reading research seems a natural back- 
drop for research involving the development of 
comprehensible texts because it explicitly con- 
siders processes such as the interaction of a 
reader's knowledge with the way in which the 
text is presented. This process orientation pro- 
vides more access points from which to con- 
sider reader/text interactions and the places 
where they may break down. 

Although recent cognitive theory and re- 
search would seem to have clear applications 
for producing more comprehensible texts, these 
notions have not yet been applied directly in the 
development of textbook discourse. Hence, a 
major motivation for the present study was to 
apply understandings gained from cognitive 
theory and research to the revision of text pre- 



sented in elementary social studies textbooks 
and to determine the effectiveness of the revi- 
sions. 

Focusing on the processes involved in 
reader/text interactions provided the means to 
pursue an additional goal in the present study, 
which was to expose the reasoning underlying 
revisions made to the texts. In past research on 
text, the creation of revisions that improve com- 
prehension has met with some success. How- 
ever, communicating the thinking that motivated 
the revisions has proved to be a more intractable 
problem. The descriptions of the revisions 
made to improve text comprehension have most 
frequently been global and intuitively obvious, 
such as “clarifying the content by highlighting 
main ideas,” “adding cohesive ties,” “providing 
background information,” “deleting irrelevant 
information,” and “clarifying the structure” 
(Beck, McKeown, Omanson, & Pople, 1984; 
Britton, Van Dusen, Gulgoz, & Glynn, 1989; 
Duffy et al., 1989; Graves et al., 1988). Al- 
though most people would agree that such de- 
scriptions characterize more comprehensible 
discourse, these descriptions in themselves do 
not serve to explain why and how specific revi- 
sions were made. The general nature of these 
descriptions of revisions may have been suffic- 
ient for the aim of the studies in which they 
were developed, which was to examine the ef- 
fects of text revisions on comprehension, rather 
than to communicate why various text features 
might cause difficulty and how to approach 
solutions. But such general statements cannot 
effectively guide the creation of more 
comprehensible text. 

In addressing the problem of how to de- 
scribe revisions in more effective ways, Duffy 
et al. (1989) suggest that what is needed is a li- 
brary of examples of text problems and their so- 
lutions. Although we believe that this notion is 
certainly a step in the right direction, it does not 
go far enough. Sets of examples need to be ac- 
companied by in-depth descriptions of the 
process of getting from the original to the revi- 
sion— that is, the identification of problematic 
aspects of the original text and their potential ef- 
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fects on target readers as well as potential solu- 
tions to the problems and explanations for why 
such solutions might work. 

Britton et al, (1989) suggest that this type 
of in-depth description of the reasoning behind 
the revision process may be difficult to obtain. 
They examined the revisions made by five sets 
of revisers, the effects of the revisions on com- 
prehension, and the rationales the revisers gave 
for making the revisions, Britton et al. con- 
cluded that, although a majority of the revisers 
made effective revisions to texts, they were less 
able to describe the kinds of changes they made 
and what motivated the changes. 

The reasoning that underlies successful re- 
visions is a rich resource that remains untapped. 
Exposing this reasoning might provide greater 
access to an understanding of why some textual 
features are problematic and a more detailed 
conception of what characterizes comprehensi- 
ble text. Thus, it was our intention to capture 
and articulate the thinking that went into the 
identification of potential problems and the de- 
velopment of solutions. 

The approach we have taken coincides with 
recommendations that Britton et al. (1989) de- 
veloped for moving toward more effective de- 
scriptions of revisions. They suggested that 
educational psychologists should develop ex- 
pertise in revising, to which they can then apply 
their “well developed vocabulary and store of 
relevant concepts about language, cognitive 
processes, and teaching” (p. 13) toward making 
knowledge of the revising process explicit. 

Thus, the goals of the present study were 
(a) to make revisions in a sequence of textual 
materials from a fifth-grade social studies text- 
book based on a cognitive processing perspec- 
tive, (b) to describe the theoretical motivations 
for those revisions, and (c) to demonstrate the 
effects of the revisions empirically by examin- 
ing students’ comprehension of the original and 
revised materials. 

A text-processing approach to revision 

The aspects of cognitive theory and reading 
research that guided the revisions focused on 
the complexity of the processes involved and on 
attributes of texts that can cause people diffi- 
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culty in the course of reading. According to a 
cognitive text-processing view (see. e.g.. Just & 
Carpenter, 1987; Perfetti, 1985). as a reader 
moves through a text, he or she encodes explicit 
text information and combines it with knowl- 
edge of word meanings and language conven- 
tions, knowledge about the form of texts, and 
general knowledge related to the content. The 
reader must identify each new piece of text in- 
formation and decide how it relates to informa- 
tion already given (see Clark. 1977; Clark.& 
Haviland, 1977). As reading proceeds, the 
reader forms a representation of the text s mes- 
sage and continuously updates it as subsequent 
text information is encountered. The represen- 
tation is formed by drawing connections both 
between outside knowledge and text informa- 
tion and between given and new text informa- 
tion. If the reader fails to bring knowledge to 
bear on the text, or if features of the text inhibit 
the readers ability to make connections be- 
tween given and new information, comprehen- 
sion may be impeded. 

A reader's failure to bring appropriate 
knowledge to bear may arise from lack of spe- 
cific knowledge of word meanings (Beck et al., 
1987; Omanson, Beck, McKeown, & Perfetti, 

1984) , lack of knowledge related to the topic 
(Beck. Omanson, & McKeown-, 1982; Pearson, 
Hansen, & Gordon, 1979), or from failure to 
recognize that particular knowledge is relevant 
to the text and needs to be accessed (Anderson. 
Reynolds, Schallert, & Goetz, 1977). 

The failure of a text to assist a reader in 
making connections may be the result of partic- 
ular text features, such as the use of references 
that are ambiguous (Frederiksen, 1981) or dis- 
tant (Cirilo, 1981; Lesgold. Roth, & Curtis, 
1979) or indirect (Haviland & Clark, 1974; Just 
& Carpenter. 1978), the failure to provide infor- 
mation that allows a reader to activate an appro- 
priate context for the text content (Anderson. 
1978; Bransford & Johnson, 1973; Kieras. 

1985) , the lack of clear connections between 
events (Black & Bern. 1981; Kintsch. Mandel, 
& Kozminsky. 1977; Stein & Nezworski. 
1978), the inclusion of irrelevant events and 
ideas (Schank, 1975; Trabasso et al., 1984). or 
the introduction of a high density of concepts 
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(Kintsch & Keenan, 1973; Kintsch, Kozminsky, 
Streby, McKoon, & Keenan, 1975). 

Our approach to revisions involved an at- 
tempt to simulate the process of the interaction 
between a reader— in this case, a 10-year-old— 
and a text. Thus, texts were evaluated by con- 
sidering how each new piece of text information 
might be handled, the kind of knowledge that 
the reader would need to bring to bear, and how 
the developing text representation would be in- 
fluenced. We hypothesized the points at which 
the process might break down (e.g., where req- 
uisite background information might be lacking 
or where a referent might be difficult to track), 
and ways in which an ideal reader might repair 
such breaks. These potential repairs were used 
as the basis for the revised version of the text. 
As we considered what might happen within a 
10-year-old reader’s comprehension process, 
we switched back and forth between (a) hypoth- 
esizing what the student might be thinking, (b) 
considering what seemed to be intended by the 
text, and (c) constructing text statements that 
might bring about understanding of the intended 
meaning for.the target reader. 

Method 

Subjects 

The subjects for this study were 40 fifth- 
grade and 45 fourth-grade students drawn from 
two elementary schools in a small, middle-class 
public school district in the northern United 
States. All students in both grades were invited 
to participate. Parental permission was received 
for 95 percent of the fifth-grade and 82 percent 
of the fourth-grade students. All students for 
whom written permission was received were in- 
cluded as subjects. 

National percentile rank scores were ob- 
tained for both the fourth- and the fifth-grade 
students on reading comprehension from the 
Metropolitan Achievement Test (Prescott, 
Balow, Hogan, & Farr, 1984). These scores 
were comparable across grade levels because 
both groups’ Metropolitan Achievement Test 
scores were obtained from tests administered 
during the month of March in each student's 
fourth-grade year. The mean comprehension 



score was 63.3 (SD = 21.9) for fifth-grade and 
60.1 ( SD = 22.7) for fourth-grade students. 

The original text materials were presented 
to the fifth-grade students in November; the re- 
vised materials were presented to the fourth- 
grade students in May of the same school year. 
The fifth-grade students had to be tested in No- 
vember because they had just completed the 
chapter in their social studies textbook about the 
exploration and colonization of North America 
and were ready to begin the chapter from which 
the materials in this study were drawn. From 
that point on in the school year, it was no longer 
possible to use these materials with fifth-grade 
subjects because the content would have already 
been covered in their classes. Thus, the revised 
materials were presented to fourth-grade stu- 
dents late in the school year. 

Materials 

The materials for the study included a se- 
quence of four passages drawn directly from a 
fifth-grade social studies textbook (Silver Bur- 
den, 1984) and a revised version of each of 
those passages. Each passage d^alt with an 
event or issue that was central to the develop- 
ment of the American Revolution. The. topics 
covered were (a) the French and Indian War, (b) 
the issue of “no taxation without representa- 
tion,” (c) the Boston Tea Party, and (d) the Intol- 
erable Acts. 

The text revisions made in the present study 
were rooted in an analysis of current elementary 
social studies textbooks recently completed by 
Beck, McKeown, and Gromoil (1989). That 
analysis suggested that the texts presented in 
many social studies textbooks, including those 
used in the present study, were likely to be 
problematic for students* learning of the in- 
tended content. 

Revised text 

In this section we discuss our goals in re- 
vising the passages under consideration and 
present in detail the revisions made to the pas- 
sage on the French and Indian War. Because of 
space limitations, the revisions made to the 
three subsequent topics will only be summa- 
rized. 
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Goals. The general goal of the revisions 
was to create a text that would assist a reader in 
connecting pieces of text information and com- 
bining that information with knowledge to de- 
velop a coherent text representation. To meet 
this goal, the revised text was based on a causal 
sequence of events leading to the Revolution, 
and presented the information in such a way as 
to expose the reasoning that connects a cause to 
an event and an event to a consequence. This 
type of presentation is what we have termed a 
causal /explanatory style (Beck et al.. 1989). 

The content goal that we inferred from the 
original textbook sequence was to provide an 
initial understanding of some of the major 
events leading to the American Revolution. This 
goal, and the topics that the textbook authors 
had selected to achieve this goal, were main- 
tained in the revision. These aspects of the orig- 
inal presentation were maintained because we 
viewed the sequence of topics presented in the 
textbook to be reasonable in that it contained 
sufficient raw' material to create a coherent se- 
quence of how and why the Revolution oc- 
curred. 

The goal of creating a causal/explanatory 
text based on this sequence was. however, con- 
strained by experimental concerns. That is. be- 
cause the purpose of the study was to compare 
students* comprehension of the original and re- 
vised versions, a primary consideration was to 
maintain a common core of content on which to 
base our evaluation of student understanding. 
An additional concern was that the revised text 
not be excessively longer than the original. Al- 
though the inclusion of causal/explanatory ma- 
terial did necessitate some increase in length, 
only the content common to both texts was 
scored. 



Revisions to the passage on the French and 
Indian War. Most U.S. history books, including 
high school and college texts, as well as three of 
the four elementary texts analyzed in the earlier 
study (Beck et al.. 1989). describe the chain of 
events that led to the American Revolutionary 
War as having begun with the French and In- 
dian War. The connection between the French 
and Indian War and*. the Revolution was that, as 
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a result of the French and Indian War. several 
sources of tension developed between the Brit- 
ish and the American colonists. In particular, 
the cost of the French and Indian War left Brit- 
ain in poor financial condition. In order to 
defray the cost of the fighting, the British Par- 
liament chose to tax the colonists. In the colo- 
nists* view, such taxes were not legitimate. 
Thus, a consequence of the war- the imposition 
of taxes -became an important initiating event 
in the conflict between Britain and the colonies. 

Given that initiating events of the Revolu- 
tion are anchored in a consequence of the 
French and Indian War, in order to set in motion 
the sequence of events that led to the American 
Revolution, students need to know that this war 
existed. But, in terms of the established instruc- 
tional goal, they do not need to understand de- 
tails of issues and events of the French and 
Indian War itself. It seems that the authors of 
the original text had three major goals in de- 
scribing the French and Indian War: (a) es- 
tablishing that the French and Indian War 
happened, (b) noting the colonists* perspective 
on the outcome of the war. and (c) foreshadow - 
ing the American Revolution. These are reason- 
able purposes that can capture w'hat young 
students need to know about the French and In- 
dian War in order to connect it to the sequence 
of events that led to the Revolution. We found 
the text lacking, however, in meeting these min- 
imal purposes. 

The general operations used in making re- 
visions were clarifying, elaborating, explaining, 
and providing motivation for important infor- 
mation. and making connections explicit. How- 
ever. in the description that fol low's we go 
beyond other descriptions of revisions by pro- 
viding instantiations of these general heuristics 
in the form of process-annotated examples that 
tie research findings to the identification of text 
problems and formulation of solutions. Because 
wc view the development of revisions as inex- 
tricably intertwined with the particular content, 
our analysis of each sentence or set of related 
sentences of the original text (O) and the solu- 
tions to the problems as manifested in the re- 
vised text (R) arc presented. 

The first sentence in the original text was 
the following: 
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(01 ) In 1 763 Britain and the colonies ended a 7- 

year war with the French and Indians. 

We judged this sentence as potentially problem- 
atic for young readers because -of two features. 
First, as an introduction to the French and In- 
dian War, it begins in the wrong place— the end 
of a war that has not previously been men-, 
tioned-and thus breaks the given/new contract 
(Clark, 1977; Clark & Haviland, 1977). Sec- 
ond, this sentence is extremely information- 
dense. Many concepts are communicated in 
relatively few words, a situation that can inhibit 
comprehension (Kintsch & Keenan, 1973; 
Kintsch et al,, 1975). 

We judged that the authors of this text in- 
tended for sentence 01 to impart the informa- 
tion that (a) the French and Indian War existed, 
(b) this war lasted 7 years, (c) it ended in 1763, 
(d) the British and their American colonists 
fought together on one side, and (e) the French 
and the Indians fought on the other side. The 
density of the information means that much has 
been left implicit. Of the five pieces of informa- 
tion, only the duration of the war and the date it 
ended are stated directly. Thus, the reader must 
make many inferences to construct the remain- 
ing information from the text. For example, the 
reader must use the phrase “ended a war" to 
construct the simple fact that a war occurred. 

Moreover, none of the information is elabo- 
rated or explained in a way that would allow a 
reader to connect the information to previous 
text concepts or to prior knowledge. This lack 
of connection becomes particularly problematic 
if the reader does not know who was Fighting 
whom'. One question that such a reader might 
have is why Britain and the colonies would be 
on the same side in a war. For this information 
to make sense, a young reader would need to 
know that the colonies belonged to Britain at the 
time; with this information, the reader might be 
able to reason that the colonies would be allied 
with the mother country. But there is evidence 
that many students lack this prerequisite knowl- 
edge (McKeown & Beck, 1990), so they would 
not be able to engage in such reasoning. A sec- 
ond question that might arise is why the French 
and Indians would be on the same side; readers 
who do not know the answer to this question 



cannot construct it from information presented 
previously in the text. A third possible source of 
difficulty is that the text does not clarify that 
Britain and France were the main parties in the 
conflict, and does not give any reason for their 
antagonism. 

In the revision, seven sentences were 
needed to resolve all the issues left incomplete 
in the original texts initial sentence. It took 
seven sentences primarily because, in order to 
supply the missing connections, the revision 
was cast in narrative form to replace the static 
description of the original text. (We note that 
there was only one other instance across the 
four passages where this much material was re- 
quired to resolve problems.) The sequence of 
events developed in the revised version was 
based on a set of agents and their actions, their 
goals, and the consequences of their actions 
(see, e.g., Stein & Trabasso, 1982), because 
those elements seem to represent the obvious 
minimal slots in a war schema. The first seven 
sentences in the revised version arc presented 
below: 

(Rl) About 250 years ago, Britain and France 
both claimed to own some of the same land, 
here, in North America. (R2) This land was just 
west of where the 13 colonies were. (R3) In 
1756, Britain and France went to war to see who 
would get control of this land. (R4) Because the 
13 American colonies belonged to Britain, the 
colonists fought on the same side as Britain. 
(R5) Many Indians fought on the same side as 
France. (R6) Because we were Fighting against 
the French and Indians, the war has come to be 
known as the French and Indian War. (R7) The 
war ended in 1763. 

The major aims of sentence Rl are providing a 
setting that motivates the war and establishing 
the primary agents who will be involved in the 
ensuing actions. It is such contextual or “set- 
ting" information that allows for interpretation 
of subsequent information (Stein & Trabasso, 
1982). Stating that Britain and France both 
claimed some of the same land was intended to 
activate a “conflict" schema in a readers mind— 
that when two parties claim the same thing, 
there could be trouble. Activation of a relevant 
schema can promote appropriate reader expec- 
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tations, which then allow a reader to draw rela- 
tionships between pieces of information more 
readily (Anderson, 1978; Kieras, 1985). 

We replaced the date that appeared in the 
original sentence ( 1763 ), with the wording 
about 250 years ago ; our intent was to frame 
the events in a “long time ago" setting with a 
general statement that is probably more easily 
encoded by young readers than a specific year. 
In addition to setting the time, sentence R1 also 
sets the place: here , in North America. 

There were two reasons we included sen- 
tence R2, which elaborates the location of the 
disputed land as just west of the 13 colonies. 
First, the wording keeps the colonies active for 
the reader, which is important because they 
constitute one of the two major agents of the 
actions described throughout the four passages. 
Second, the sentence was designed to confront a 
misconception found in earlier work (McKeown 
& Beck, 1990) among subjects of this age, who 
often seem to think that the French and Indian 
War was fought over ownership of the 13 colo- 
nies, and that Britain acquired the colonies as a 
result of that war. Sentence R2 explicitly places 
the disputed land beyond the colonies. 

The purpose of sentence R3 was to describe 
the action taken to attempt to resolve the dispute 
between Britain and France. It does so by di- 
rectly stating that there was a war and reinstat- 
ing from sentence R1 the reason for the war and 
the major disputing parties. 

Sentence R4 not only directly places the 
colonies on the same side as Britain, but also 
provides the reason that the colonies would be 
on Britain's side, in an attempt to make that 
partnership meaningful rather than an uncon- 
nected fact. The Because... phrase that provides 
a reason for the colonists’ alignment with Brit- 
ain was placed in the initial position in the sen- 
tence to signal the reader that a connection 
would need to be made. It has been shown that 
making such signals explicit has the potential to 
enhance comprehension (Marshall & GlCck, 
1978; Spyridakis & Standal, 1987; Zinar, 
1990). 

The purpose of sentence R5 was merely to 
establish that the Indians fought on the same 
side as France and thus enable the explanation 



READING RESEARCH QUARTERLY • 1991 XXVI/3 



:n sentence R6 about why the war has this pecu- 
liar name when it was not a war between the 
French and the Indians. Such an explanation is 
warranted by the use of the wars name as a 
heading in the original text. We note that there 
is a deliberate inconsistency in our choice to in- 
clude in the revision an explanation of the align- 
ment of the colonies with Britain, but not the 
alignment of the Indians with France. The latter 
was not provided because the explanation is 
quite complicated and seemed more likely to 
disrupt coherence of the passage than to help 
the reader, in view of the tangential role of the 
missing information for the topic at hand and 
for the overall topic of the passages. 

The apparent inconsistency in our inclusion 
of explanatory information is an apt example of 
situations in which issues in a text that seem 
alike on the surface do not necessarily warrant 
like treatment. In one case, a relationship be- 
tween allies was made explicit in order to pro- 
mote understanding of that relationship and thus 
of the text as a whole. In the other case, a simi- 
lar relationship was left implicit because under- 
standing of the relationship was not necessary 
for the content goal established and might even 
prove distracting, for young readers. Inclusion 
of information that is not directly related to the 
topic can interfere with comprehension 
(Schank, 1975; Stein & Trabasso, 1982). 

Finally, sentence R7 was included simply 
to close the French and Indian War sequence in 
order to set the stage for discussion of the con- 
sequences of the war that were relevant to the 
Revolution. At this point in the revised text, all 
issues from sentence 01 have been resolved. 

We now consider the original text's descrip- 
tion of the results of the war, w'hich was pre- 
sented in the following two sentences; 

(02) As a result of this war France was driven 
out of North America. (03) Britain wuuld now 
rule Canada and other lands that had belonged 
to France. 

The initial problem with this sequence is that 
the phrase as a result of this war is likely to lead 
readers to expect that the w inner of the war will 
be named. Instead, the text goes on to present 
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the loser, combined with the potentially unfa- 
miliar language convention driven out of North 
America. This language may fail even to com- 
municate the intent of the sentence -that France 
lost the war— much less help students infer that 
Britain won. Yet Britain’s victory, and its effect 
on the colonies, is much more significant than 
France’s loss for the purposes of setting the 
stage for pre-Revolutionary conflict. 

The revised sentences about the results of 
the war appear below: 

(R8) Britain won the war. (R9) Now Britain had 
control of North America, including Canada. 
(RIO) The French had to leave North America. 

Sentence R8 states directly that Britain won the 
uwr to fill what was assumed to be an important 
slot in young childrens war or conflict schema: 
a “winner.” With the winner established, sen- 
tence R9 tells what was gained from winning. 
This statement serves to bring closure by re- 
splving the conflict begun in the first sentence - 
that Britain and France were in a dispute over 
land. Research on the role of a narrative schema 
in comprehension supports the importance of 
providing a resolution to a conflict that has been 
set up in a text (see Stein & Trabasso, 1982). 
Sentence RIO then states what happened to the 
French, in language that is more likely familiar 
to young readers than that used in sentence 03. 

Next, the original text discusses the effects 
of the outcome of the war on the colonists: 

(04) This brought peace to the American colo- 
nies. (05) The colonists no longer had to fear 
attacks from Canada. 

These sentences present solutions to problems 
that have not been established. That peace has 
now befallen the colonies may puzzle r young 
readers, as it implies a contrast, but the con- 
trasting situation has never been presented. The 
same is the case for relief from attacks from 
Canada: The statement treats the colonists’ fear 
of attacks as given information, but it has not 
been presented in the text. Thus, the reader en- 
counters new information, but does not have the 
given information in memory to which it can be 
connected. 



The revised text’s discussion of the effects 
of the outcome of the war on the colonists takes 
into consideration the role of given and new in- 
formation in comprehension (Clark, 1977; 
Clark & Haviland, 1977) by including the in- 
formation that should have been given: 

(R1 1) The colonists were very glad that Britain 
had won. (R12) They now felt safer in their 
homes. (R13) Before the war, Indians had often 
attacked colonists who lived near the borders. 
(R14) Now Britain owned these lands where the 
Indians lived. (R15) The colonists were sure 
that Britain would protect them. 

After the colonists’ reaction to the outcome of 
the war is presented in sentence R1 1, sentences 
R12 through R15 explain why the colonists 
were pleased by elaborating the situation that 
existed before the war and how the war's out- 
come had ameliorated that situation. Thus, the 
revised sequence explains the situation referred 
to in sentence 05. The introductory words Be- 
fore in sentence R13 and Now in sentence R14 
were included as explicit signals, in this case to 
help readers establish temporal sequence. 

The next section of the original text fore- 
shadows the Revolutionary war: 

(06) The Americans were happy to be part of 
Britain in 1763. (07) Yet a dozen years later, 
these same people would be fighting the British 
for independence, or freedom from Great Brit- 
ain’s rule. 

That there would be a change in the positive at- 
titude expressed in sentence 06 must be in- 
ferred from the word Yet that begins sentence 
07. In our revision of these sentences, we again 
took account of the demonstrated importance of 
explicating the relationships between key pieces 
of information, and provided a direct statement 
of the change that was to occur: 

(R 16) The colonists were happy to be a part of 
Britain, but that was about to change. (R17) 
They began to decide that they would rather 
have their own country, independent from Brit- 
ain. 

Sentence R16 forecasts that the colonists’ posi- 
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tive attitude was about to change. Sentence R 1 7 
then provides an explanation of what motivated 
the change. It is the colonists' motivation of 
wanting to have their own country that provides 
a context for the major issue of the subsequent 
topics. This revision is another reflection of the 
importance of supplying a context that will al- 
low a reader to interpret subsequent informa- 
tion. 

The final portions of the original and re- 
vised texts were as follows: 

(08) This war was called the War for Indepen- 
dence. or the American Revolution. (09) A rev- 
olution changes one type of government or way 
of thinking and replaces it with another. 

(R18) So a dozen years later, the colonists 
would be fighting the British for freedom from 
Great Britain's rule. (R19) This later war would 
be called the War for Independence, or the 
American Revolution. 

The word So in the initial position in sentence 
R18 is another explicit connector, which was 
provided in this case to help the reader relate the 
motivation described in sentence R17 to the up- 
coming fight. In addition, in place of the vague 
referent in sentence 07 to these same people , 
sentence R18 states explicitly that it is the colo- 
nists who will be fighting the British. Sentence 
R19 uses the phrase this later war and the sub- 
junctive future tense would be called to help 
young readers distinguish between the upcom- 
ing war for independence and the topic of the 
foregoing text, the French and Indian War. 
These changes are supported by Frederiksen's 
finding that young readers need greater lexi- 
cal and semantic explicitness (1978), and by 
Lesgold et al.'s study of the usefulness of certain 
text mechanisms for referring to previously 
given information (Lesgold et al., 1979). In re- 
vising the final sentence in this passage, we de- 
cided to omit the definition of revolution, as we 
believed that any attempt to define it was likely 
to be unsuccessful in communicating the con- 
cept, which in any case was not necessary to un- 
derstand the central concepts in the passage. 
(For ease of comparison, the original and re- 
vised versions of the French and Indian War text 
appear in their entirety in the Appendix.) 



u No taxation without representation For 
the three remaining passages, our analyses of 
the goals of the text, the presentation of the 
message, and the effectiveness of communicat- 
ing that message to young students were similar 
to those described for the passage on the French 
and Indian War. We then used the same perspec- 
tive described above to create revisions that 
would alleviate the problems identified. We will 
not present the details of those revisions, but we 
will summarize our revisions to these passages. 

The section of the text on the topic of “No 
taxation without representation" describes Brit- 
ain’s taxation of the colonies after the French 
and Indian War, and the colonists' opposition to 
that taxation. Understanding why the colonists 
found taxes to be such an important issue is cru- 
cial in the development of the chain of events 
that brought about the Revolution. And such an 
understanding requires that students have some 
notion of what it means to be represented in a 
government body. The problem with the origi- 
nal text is that the discussion of representation is 
virtually devoid of explanatory power. 

The notion that the colonists did not have 
representation regarding taxation has two roots: 
(a) that they could not vote for members of Par- 
liament, and (b) that the colonial assemblies, 
which were elected by the colonists, had no 
voice in British taxation policies. In the original 
text, an understanding of these two issues would 
have to be constructed from the following sen- 
tences: 

(01) The British lawmaking body was and stiil 
is called Parliament. (02) The colonists were 
not members. (03) The British started passing 
laws to tax the colonies.... (09) The colonists 
got very upset about these taxes. (010) Their 
own colonial assemblies had not voted for them. 
(Oil) They did not welcome Parliament’s tax 
laws. 

For students to understand the representation is- 
sue, they would have to interpret sentences 01 
and 02 to mean that Parliament was an elected 
body, but that the colonists could not take part 
in these elections. They would need to interpret 
sentences OIO and 01 1 to mean that the col- 
onial assemblies, as the colonists' elected gov- 
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crnmcntal bodies, had no say in determining 
British tax policy, and that therefore the colo- 
nists did not feel that they had to follow British 
tax laws. It was our judgment that readers 
would be unable to make either interpretation 
because the presentation of these issues is so 
brief and opaque. 

For the revised text, we chose to focus 
solely on the colonists lack of a voice in Parlia- 
ment's decisions, because we saw it as a more 
direct path to understanding what the colonists 
meant by “No taxation without representation." 
Thus, the major revision we made in this sec- 
tion was to develop the following explanation, 
in which representation is described as having a 
say in what the government does: 

...[T]hc British Parliament made the tax laws 
that the colonists had to follow, but the colonists 
did not have any say in deciding them. “Having 
a say" in making the laws means that you de- 
cide. by voting, for the people who make the 
laws. The people who are elected to make the 
laws arc called representatives. The problem 
was that the colonists were not allowed to vote 
for the people who made the laws, so they had 
no representatives in Parliament. When elec- 
tions were held in Britain, only the people over 
in Britain voted. The colonists felt that since 
they could not vote for the people who make the 
laws, they should not have to follow the laws. 
So their motto became "No taxation without 
representation." 

This revision focused on the explanation of the 
issue of having a say in government and how 
that issue was connected to the colonists' reac- 
tion to the taxes. In addition, a variety of small 
changes were made to establish local coher- 
ence. 

The Boston Tea Part}'. The information this 
passage provides is that (a) the British kept a 
small tax on tea. (b) the colonists were angry 
about the tax and boycotted British tea. and (c) 
even after Parliament lowered the price of tea, 
the colonists refused to buy it and responded 
with protests, one of which was throwing tea 
from a ship in Boston Harbor. The original text 
presents all of the actual events, but the motiva- 
tions for those events and how they relate to one 



another are conspicuously absent. For example, 
the text does not tnake explicit that, even after 
the price of tea was lowered, the tax on the tea 
remained. 

Most of the revisions for this passage thus 
involved explaining and providing motivations 
for actions and reactions, and explicitly con- 
necting causes to events and events to conse- 
quences. For example, in the revised version 
the colonists* anger over the tax is portrayed ex- 
plicitly as the cause of their refusal to buy tea. 
which in turn was presented as the motivation 
for the British government's reduction in the 
price of tea. That the tax remained despite the 
lower price was stated and discussed as the 
cause of the colonists' strong reaction to the new 
situation. As an illustration of the implementa- 
tion of these connections, consider the follow- 
ing excerpts from the original and the revised 
texts: 

Original: Boatloads of tea were sent to Amer- 
ica. Since it was cheaper than ever, the British 
thought that surely the colonists would buy tea 
now! They were wrong. Tea was burned. Tea 
was left to rot. 

Revised: Since it was now cheaper than ever, the 
British thought that surely the colonists would 
buy tea! So they sent boatloads of it to the colo- 
nies. But. because the tea still had the tax on it. 
the colonists were as angry as ever. To show 
their anger, the colonists burned some of the 
tea. They left some to rot. 

Thus, the revisions in this section were de- 
signed primarily to help readers understand the 
colonists' actions as motivated by their objec- 
tion to the taxes. 

The Intolerable Acts. This final section 
presents the information that the British re- 
sponded to the Boston Tea Party with a punish- 
ing set of laws, called the Intolerable Acts. 
These acts included closing the port of Boston, 
disallowing self-government, and requiring the 
quartering of troops. Colonists in Boston al- 
erted other colonies to the situation, which 
brought an influx of help and supplies and 
spread fear of similar retribution among the 
other colonics. The major problem with the 
original passage is that it does not explain the 
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severity and significance of the punishments. 
The text simply says that the British passed 
what the colonists called the Intolerable Acts 
and that intolerable means “unbearable.” The 
motivation for the acts is stated as being to pun- 
ish the people of Boston, but the acts themselves 
are merely listed as follows: 

The port of Boston was closed. No self govern- 
ment was allowed in Massachusetts. British 
troops had to be housed and fed by the Massa- 
chusetts colonists. 

In order to understand how the Intolerable 
Acts and the colonists’ reaction to them were 
linked to the other events leading to the Revolu- 
tion, a reader needs to understand the conse- 
quences of these Acts for the colonists. Thus, 
the major revisions involved explaining the se- 
verity and significance of each of the Acts. Con- 
sider just the discussion in the revised text of the 
Act that closed the port of Boston: 

These laws did many things that made life in 
Boston very difficult. For example, the British 
closed the port of Boston. That meant no ships 
could get in or out. Since most of Boston's sup- 
plies came in by ship, food and other important 
things soon became very hard to get. 

The other colonies' reaction to the plight of Bos- 
ton is also an important component of the major 
chain of events, but the original text is rather 
obtuse about why the other colonies were con- 
cerned. Consider the original and revised treat- 
ments of this issue: 

Original: People wondered if their colony 
would be the next to feel the anger of Great 
Britain. 

Revised: It frightened people in other colonies 
that Britain could act so harshly. People feared 
that their colony might be the next to feel the 
anger of Great Britain. 

The revisions to this section, which focused on 
explaining the harshness of the Intolerable Acts 
and the colonists’ reaction to them, were in-w 
tended to prepare readers to understand the ex- 
tent to which the conflict with Britain was 
intensifying. 



Characteristics of all revisions. As noted 
earlier, the text revisions were designed to pro- 
vide causal connections that were generally not 
oifered in the original text. Thus, the revision 
was longer than the original; across all four pas- 
sages, there were 583 words in the original text 
and 1 ,046 words in the revised text. However, 
students' comprehension was compared only on 
content that was common to both versions. 
Readability calculations using the Fry Readabil- 
ity Scale (Fry, 1975) yielded an average read- 
ability of 6.0 for the original text and 7.6 for the 
revised version. 

Measures 

To assess comprehension of these materi- 
als, we used both free recall and open-ended 
questions. Because young students tend to give 
limited recalls, particularly on relatively unfa- 
miliar content, the questions that were devel- 
oped for each passage were extensive. The 
same sets of questions were presented for the 
original and revised texts. The questions fol- 
lowed the sequence of the text and tapped 
knowledge of states, events, and relations be- 
tween them. (The questions for the French and 
Indian War segment are shown in the Appen- 
dix.) 

Procedure 

The text materials were presented to each 
student in individual sessions with an examiner 
(one of the authors). The examiner followed a 
written script that included a brief introduction, 
instructions, recall prompts, and short-answer 
questions. Each student read the text silently in 
four sections (corresponding to the four topics 
discussed above), one section at a time. The 
student was asked to think of the sections as 
passages from the student's social studies text- 
book. After completing each section, the stu- 
dent was asked to tell in his or her own words 
what the section was all about and to respond to 
a series of open-ended questions. The recalls 
and answers to questions were recorded on 
audiotape and later transcribed for scoring. 

The only difference in procedures between 
the two groups was that the fourth-grade stu- 
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dents were presented with a very brief overview, 
lasting about 3 minutes, of some of the content 
that the fifth-grade students had studied over a 
2-month period. This step was taken because 
the fifth-grade students had studied content that 
the textbook authors apparently intended as pre- 
requisite for learning about the American Revo- 
lutionary period, and the fourth-grade students 
had not. Specifically, the examiner showed the 
fourth-grade students an outline map that 
showed the North American continent and Eu- 
rope. Following a prepared script, the examiner 
pointed to the map as she told subjects that, a 
long time ago, people from England and France 
had come over to North America to explore the 
land. She indicated where the French had ex- 
plored and settled, where the British had ex- 
plored and settled, and the general location of 
the 13 British colonies. 

Scoring 

Scoring of the recalls. In order to compare 
students' recalls of the original and revised pas- 
sages, we first analyzed the text into idea units 
to be used as a basis for scoring. Each of the 
four passages was divided into units based on a 
narrative analysis procedure developed by 
Omanson (1982) and used in previous work 
(Beck et al., 1982, 1984). Omanson's method 
involves analyzing clauses that portray an event 
or state into content units and the relations that 
connect these units. 

In the present work, we adapted Omanson s 
procedure in order to use it with expository 
text, resulting in units that were often smaller 
than a clause. In the original procedure, a con- 
tent unit is defined as portraying a single state 
or event. But because expository text is often 
more conceptually dense than narrative text, a 
single state or event may contain several chunks 
of information. For example, in Omanson's 
original procedure, the sentence Britain would 
now rule Canada and other lands that had be - 
longed to France would be considered a single 
unit because it portrays a single state in terms of 
one main clause and a restrictive clause. In the 
present study, this sentence was divided into 
four units because it was determined that it con- 



tained four chunks of information: 

1 . Britain would now rule. 

2. Canada would be under this rule. 

3. Other lands would also be under this rule. 

4. The land that would be ruled had once be- 
longed to France. 

The procedures for developing idea units 
for use as the scoring base involved first divid- 
ing the text into clausal units, then separating 
from the main clause any phrases that repre- 
sented distinct states, events, locations, and 
qualifiers. Compound subjects and objects were 
separated, and separate units were designated 
for any complement that added content. The re- 
sulting units were generally smaller than 
clauses, but larger than propositions. 

Because the elliptical nature of the original 
text required an inordinate number of infer- 
ences, we represented these in the scoring base 
as implied units. The criterion we used was to 
include as implied units only information that 
was deemed necessary for understanding. For 
example; for the text statement In 1763 , Britain 
and the colonies ended a 7-year war with the 
French and Indians , we included the implied 
units colonists with Britain and Indians with 
France in our scoring base because a basic 
piece of information about a war is to know who 
is fighting whom, even though that information 
is not explicitly stated. In contrast, the informa- 
tion that the war began in 1756, which could be 
derived from the sentence, was not given status 
as an implied unit because it was not important 
for understanding the central events and states 
in the text. 

To determine reliability, 30 percent of each 
text was independently divided into units by a 
second rater (another of the authors). An in- 
terrater reliability of .92 agreement was ob- 
tained. Disagreements w'ere settled by mutual 
agreement. This procedure resulted in the iden- 
tification of 145 idea units in the original text 
and *170 idea units in the revised text. For pur- 
poses of scoring, 124 units of common content 
were established between the two texts, and 
only the common content was scored. Each stu- 
dent was given credit for a unit if his or her re- 



118 



115 



Volume 2, Spring 1997 



Subject-Area Learning 



reading research quarterly • 1991 XXVI/3 



call contained the gist of that unit. Scoring was 
again done by one of the authors, and interrater 
reliability on 20 percent of the recalls scored in- 
dependently by a second rater was .91 agree- 
ment. Disagreements were settled by a third 
rater. 

Scoring of the question responses. In order 
to compare performance on the comprehension 
questions by students who read the original and 
students who read the revised text, we first de- 
veloped a template of model answers to the 
questions. Of the 34 questions across the four 
segments, 28 were scored either 0 or 1. Half 
credit was possible for the remaining 6 ques- 
tions. Because of the open-ended nature of the 
questions, reliability of the scoring was calcu- 
lated. Like the recalls, the questions were 
scored by one of the authors: interrater reliabil- 
ity for 20 percent of the answers scored by a 
second rater was .94 agreement. Disagreements 
were settled by a third rater. 



Quantitative results 

In this section we present the results of the 
statistical analyses of comprehension perform- 
ance on the original and revised text. The pri- 
mary question posed by this study was whether 
the revised text produced better comprehension 
than did the original text. The data were ana- 
lyzed to compare the number of common units 
recalled and the number of questions answered 
correctly by students who read the original and 
students who read the revised text. 

Overall efTect on comprehension 

The data for recall of units common to both 
texts and the scores for answers to questions 
were examined in separate, mixed-design 2x4 
analyses of variance (ANOVAs), with text type 
(original versus revised) as the between -subject 
factor and with passage segment (on the French 
and Indian War, "No taxation...," the Boston 
Tea Party, and the Intolerable Acts) as a within- 
subject factor. 



Table 1 shows the mean percentage of con- 
tent units recalled by students who read the 
original and revised texts. The table indicates 
that students who read the revised passages re- 
called more units from all four sections, and the 
total difference for all passages combined was 
significant. F(l. 83) = 4.67,/? < .05. 

The advantage in comprehension perform- 
ance for students who read the revised text is 
even clearer in their answers to questions. Table 
2 shows the mean percentage of questions an- 
swered correctly by both groups. Again the stu- 
dents who read the revised texts answered more 
questions coriectly for all four passage seg- 
ments than students who read the original, and 
the difference, for all passages combined was 
significant. F(l. 83) = 15.10,/? < .01. 

Differential effects for four passage 
segments 

Another question of interst in this study 
was whether there would be a difference in 
comprehension performance across the four 
passages. Specifically, we hypothesized that 
performance on the Boston Tea Party segment 
might be superior to performance on the other 
segments, both because of its more narrative 
style and because it was more likely that stu- 
dents would have encountered the information 
previously, in stories in trade books. Results 
showed that there were indeed significant main 
effects of passage segment on both percentage 
of content units recalled. F{ 3. 249) = 48.72. p 
<.01. and percentage of questions answered 
correctly. F( 3. 249) = 21.98, p <.0l. There 
was no significant effect of the interaction be- 
tween text type and passage segment (p > .05). 
As shown in Tables 1 (for recall) and 2 (for 
questions), both the original and revised groups 
showed the highest comprehension perform- 
ance on the Boston Tea F^rty segment and the 
lowest performance on the "No taxation" seg- 
ment. It is not surprising that students had the 
most trouble with the "No taxation" text because 
it contains a very difficult concept, that of rep- 
resentative government . and there is evidence 
that students of this age have severely limited 
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Tablle 1 Means and standard deviations for percentage of content units recalled by students 
who read the original text and students who read the revised text 







Original 


version 


Revised version 


Passage segment 


Total ft of units 


M 


SD 


M 


SD 


The French and Indian War 


38 


11.3 


8.5 


14.6 


12.1 


No taxation without representation 


38 


8.3 


8.0 


13.0 


11.0 


The Boston Tea Party 


27 


21.6 


12.2 


26.6 


12.0 


The Intolerable Acts 


21 


13.1 


11.5 


17.9 


15.5 


Total 


124 


13.6 




18.0 





Table 2 Means and standard deviations for percentage of questions answered correctly by 
students who read the original text and students who read the revised text 







Original version 


Revised version 


fcssage segment 


Total # of questions 


M 


SD 


M 


SD 


The French and Indian War 


8 


30.9 


26.0 


50.8 


27.1 


No taxation without representation 


9 


22.4 


24.2 


38.3 


26.5 


The Boston Tea Rarty 


8 


41.9 


31.0 


61.7 


23.1 


The Intolerable Acts 


9 


24.3 


29.3 


45.6 


33.0 


Total 


34 


29.9 




49.1 





notions about this concept (McKeown & Beck, 
1990). 

The two dependent measures (recall and 
questions) reveal the same pattern of specific 
performance for the four passages. Given the 
limited amount of information young students 
provide in recall protocols, it is important that 
the question data support the recall findings. 



Qualitative results 

It is clear from the quantitative analyses 
that there were differences in the amount of text 
recalled and the number of questions correctly 
answered between the students who read the 
original and the students who read the revised 
text. Although these quantitative results reveal 
the presence of differences, they cannot show 



the nature of the differences. Thus, to investi- 
gate the nature of the differences between the 
representations students developed from read- 
ing the two different texts, qualitative methods 
were developed to analyze students’ recall pro- 
tocols and their answers to questions. 

Analysis of prototypical recalls. The goal 
of the qualitative analysis of recalls was to por- 
tray any group differences in the nature of the 
information recalled from the two texts. It is 
difficult to compare the information recalled 
from two versions of a text because differences 
in wording between the texts are likely to result 
in differences in wording in the recalls. To over- 
come this problem, we developed prototypical 
recalls using language common to both texts. 
These prototypical recalls are models that re- 
flect the recall of a typical student in each 
group. The models were developed from the set 
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of units most commonly recalled by students 
who read the original and students who read the 
revised texts. This method of examining recall 
patterns has been used in previous studies (Beck 
et al., 1984; McKeown, Beck, Omanson, & 
Pferfetti, 1983) and is similar to a method used 
by Spilich, Vesonder, Chiesi, and Voss (1979; 
see also Omanson, Beck, Voss, & McKeown, 
1984). 

The process of developing these prototypi- 
cal recalls involved several steps. For each 
group, we first determined which of the units 
common to both texts were recalled by at least 
15 percent of the students in that group. We 
chose 15 percent because this criterion resulted 
in the inclusion of enough units in the prototypi- 
cal recalls to show differences between the two 
groups. (Although 15% was the minimum crite- 
rion for inclusion, some of the units were re- 
called by as many as 70% of the students.) The 
units that met the criterion were then assembled 
into paragraphs, adding only the wording nec- 
essary to form complete sentences. Because the 
purpose of these analyses was to allow compari- 
son of the recall of the two groups, only the 
wording of the units as they had appeared in the 
revised text was used. 

Analysis of question responses . The pur- 
pose of the qualitative analysis of question re- 
sponses was to examine patterns of differences 
in students’ understanding that might not have 
been revealed in the recalls. Because young stu- 
dents’ recalls are typically rather sparse, the 
questions were intended to elicit students’ ideas 
about important concepts from the text that they 
might not have included in their recalls— either 
because they did not remember at the moment 
or because they did not understand. Thus, the 
qualitative analysis of questions can provide a 
sense of the extent to which the students under- 
stood the main issues of the passages, and 
where their understanding went awry. To cap- 
ture both understanding and misunderstanding, 
we examined the content of each question re- 
sponse and developed categories based on pat- 
terns that emerged. 

In some cases, the categories were quite 
straightforward. For example, in the segment 
on the French and Indian War, responses to the 
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question Who was fighting against w hom in the 
war? were simply grouped according to the par- 
ties named. In other cases, however, we had to 
interpret the focus of the response and how- it 
was related to the question. For example, for 
the question What does it mean that the colo- 
nists were not members of Parliament ? we iden- 
tified three categories of responses representing 
a range of understanding. The first category, 
which represents the fullest understanding of 
the target concept, included responses that dis- 
cussed voting for government officials or hav- 
ing a voice in government (e.g., “They didn't 
get to vote for who they wanted to think up the 
laws"). The next category comprised responses 
that focused on the tax issue (e.g., “They 
weren’t used to the taxes"). A third category in- 
cluded responses that merely mentioned the 
wording of not being members . but gave no in- 
terpretation to that phrase (e.g., “They didn’t 
belong to it"). 

In the following sections, we present a 
comparison of the prototypical recalls for stu- 
dents who read the revised text and students 
who read the original text. A similar compari- 
son is presented for the question responses. The 
analyses are presented by text segment because 
students were presented with one segment at a 
time, followed by the recall and questions for 
that segment. The question responses selected 
for discussion are those that portray a character- 
ization of students’ understanding of the main 
concepts and that capture the differences in un- 
derstanding between the two groups. 

The French and Indian War 

Prototypical recalls. The prototypical re- 
calls for the segment on the French and Indian 
War are shown in Table 3. Students who read 
either version typically recalled that Britain and 
France fought against each other, that Britain 
gained some control, and that the American 
Revolution was to follow. As shown in the table, 
students who read the revised text were likely to 
recall two important additional pieces of infor- 
mation: the motivation for the war (control of 
the land) and an outcome of the war (the colo- 
nists felt they would now be safe). 
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Table 3 Prototypical recall protoc ols for the French and Indian War text segments 
Origin 31 text Revised text 



It was about the French and Indian War. In 1763. 
Britain and France had a war. Many Indians fought 
on the same side as France. Now Britain had con- 
trol. A dozen years later the colonists would be 
fighting for freedom. This war would be called the 
War for Independence or the American Revolution. 



It was about the French and Indian War. Britain and 
France had a war to see w ho would get control of the 
land. Many Indians fought on the same side as 
France. The \oor ended in 1763. Britain won the 
Miar. Now Britain had control. The colonists felt 
safer and they were happy ; But then the colonists 
would be fighting for freedom. This later war would 
be called the War for Independence or the American 
Revolution. 



Note. The italicized units in each prototypical recall are those units that were recalled only by students who read that version of the text. 



Question responses. In general, the pattern 
of responses to questions about the French and 
Indian War segment showed that there were two 
main points of confusion: the identity of the 
agents who were involved in the war and the 
motivation for the war. In both cases, this con- 
fusion was more pronounced for the students 
who read the original text. 

First, answers to the question Who was 
fighting against whom? revealed greater confu- 
sion about the combatants on the part of the stu- 
dents who read the original text, who showed no 
evidence of knowledge that the colonists were 
aligned with Britain. Specifically, only 23 per- 
cent of students who read the original text pitted 
the British against the French and Indians, and 
not one of these students mentioned the colo- 
nists as being on the side of the British. In con- 
trast, 38 percent of the students who read the 
revised text correctly pitted the British against 
the French and Indians, and 1 1 percent of these 
students mentioned the colonists' alignment 
with Britain. The differences in understanding 
between students who read the two versions 
were more obvious in students' responses to the 
question Who won the war? Of the students who 
read the revised text, 80 percent correctly iden- 
tified Britain as the winner; in contrast, only 45 
percent of the students who read the original 
text were able to answer this question correctly. 

The motivation for the, war was a source of 
confusion for both groups, but the confusion 
was more serious for the readers of the original 



text. A correct response to Why was the war 
fought? (to gain control of the land) was given 
by 62 percent of students who read the revised 
text, but by only 35 percent of students who 
read the original version. The predominant re- 
sponse from readers of the original text was that 
the war was fought for independence (45%). At 
first it might seem that these students were con- 
fusing the French and Indian War with the colo- 
nists' upcoming fight for independence, which 
is foreshadowed in this section; however, only 
22 percent of the students who gave this re- 
sponse identified the colonists as the party seek- 
ing independence. The majority (55%) of these 
students identified variously the Indians, Brit- 
ish, French, or Canadians as the agents seeking 
independence. The confusion of students who 
read the original text over the motivation for the 
war also affected their answers to the question 
What did they get by winning? In response to 
this question, 69 percent of students who read 
the revised text correctly stated that the prize 
was land, but only 43 percent of the students 
who read the original text gave this response. 

Overall, the qualitative analyses suggest 
that more students who read the revised text 
seemed-to complete this segment of the text 
having established a context for the events to 
come. That is, they had a clearer understanding 
that North America was controlled by Britain" 
that the colonists belonged to Britain, and that 
the colonists were currently satisfied with that 
situation. 
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Table 4 Prototypical recall protocols for the “No taxation. . text segments 



Original text Revised text 



Britain wanted them to pay taxes. Parliament passed 
tax laws on glass, paint, and tea. The colonists 
thought these taxes were very unfair. They even de- 
stroyed tax collectors* homes. Britain decided to 
stop trying to collect the taxes by 1770 . 



Britain wanted them to pay taxes. These taxes were 
put on glass, paint, and tea. The colonists thought 
the taxes were very unfair. The colonists were not 
allowed to vote. They felt that since they could not 
vote for the people who make the laws, they should 
not have to follow the laws. They would not pay 
taxes to that gowmment. The colonists showed their 
anger by not buying anything that came from Brit- 
ain. They even destroyed tax collector s homes. Brit- 
ain decided to stop trying to collect the taxes. Only a 
few taxes were left. 



Note. The italicized units in each prototypical recall are those units that were recalled only b * student who read that version of the test. 



“No taxation without representation" 

Prototypical recalls. Table 4 shows the pro- 
totypical recalls for this segment of the text. As 
shown in the table, students in both groups typi- 
cally recalled that the British passed tax laws, 
that the colonists thought these laws were un- 
fair, and that the colonists reacted by protesting 
the tax laws. The additional information re- 
called by students who read the revised text was 
related to the central theme of the passage: that 
the colonists were upset about the taxes because 
they had no voice in determining the tax laws. 

Question responses. Students’ understand- 
ing of the issue of Britain's tax laws and the col- 
onists' reaction to them was captured in their 
responses to three questions, which center on 
the issue of the colonists' not having a say in 
Britain's government: What does it mean that 
the colonists were not members of Parliament? 
What does m No taxation without representation 
mean? and Why were the colonists upset about 
the taxes? In responses to the first two of these 
questions, students who read the revised text 
showed a striking advantage in understanding 
the issues underlying the colonists' discontent. 

In response to the question regarding the 
colonists’ lack of membership in Parliament, 38* 
percent of the students who read the revised text 
responded that this meant that the colonists had 
no representation (e.g., “They didn't have any- 



body to vote for and the British did**), but only 8 
percent of the students who read the original 
text gave this response. The greatest proportion 
of students in the original group (33%) indi- 
cated that membership in Parliament had some- 
thing to do with the taxes, but did not relate the 
issue to the concept of representation (e.g., 
“Since they weren't members they had to pay 
taxes"). 

In response to the direct question regardine 
the meaning of “No taxation without representa- 
tion," more than three times as many students in 
the revised group (51 %) as in the original group 
(15%) included in their response the point that 
the colonists had no voice in government. It is 
not surprising that so many more students in the 
revised text condition showed an understanding 
of representation because the revised text ex- 
plains this concept, whereas the original text 
leaves it to be inferred from very obtuse text 
statements. 

Students who read the revised text were 
thus much more able to grasp the issue that 
caused the colonists' anger and their actions of 
protest. However, when specifically asked why 
the colonists were upset about the taxes, they 
still tended to cite only financial concerns. The 
most common response both from students who 
read the original (35%) and from students who 
read the revised text (44%) was that the colo- 
nists' distress over taxes was simply a matter of 
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Table 5 Prototypical recall protocols for the Boston Tea Party text segments 

Original text Revised text 



It was about the Boston Tea Party. The British kept a 
tax on tea. The colonists* feelings about Britain's 
taxes had not changed. The colonists refused to buy 
any tea. The British greatly lowered its price. The 
British thought that surely the colonists would buy 
tea. So they sent boatloads of tea to the colonies. 
The colonists were as angry as ever. The colonists 
burned some of the tea. They left some to rot. The 
Sons of Liberty late one night threw more than 300 
chests of tea into the water. The colonists called this 
the Boston Tea ftjrty. 



It was about the Boston Tea Party. The colonists' 
feeling about Britain's taxes had not changed. The 
colonists refused to buy any tea. Britain hadn't been 
able to sell their tea. They greatly lowered its price. 
Now the price of tea hi as less than it had e\'cr been, 
but it still included the tax. The British thought that 
surely the colonists would buy tea. So they sent 
boatloads of tea to the colonies. The colonists were 
as angry as ever. The colonists burned some of the 
tea. They left some to rot. The Sons of Liberty snuck 
into Boston Harbor late one night, and threw' more 
than 300 chests of tea into the water. The colonists 
called this the Boston Tea ftirty. 



Note. The italicized units in each prototypical recall arc those units that weir recalled only by students who read that version of the text. 



money (e.g.. “cause they didn't think that they 
should have to pay extra money to buy the 
stuff*)* Only 16 percent of the revised group 
and 5 percent of the original group stated that 
the colonists were upset because they were not 
represented in the decision leading to the taxes. 

One hypothesis for this seeming inconsis- 
tency is that, even though information about the 
principle was available to students in the revised 
group, they (like the students who read the orig- 
inal text) tended to base their responses on a 
more familiar notion— that taxes mean paying 
money, and no one favors that. The concept of 
representation is much more complex and novel 
for these students, so they responded from a 
more easily accessible notion. 

The Boston Tea Party 

Prototypical recalls . Table 5 shows the pro- 
totypical recalls for the Boston Tea Party seg- 
ment. As shown in the table, students who read 
either text tended to recall that the colonists still 
felt the same about the taxes, that they refused 
to buy tea, that Britain lowered the price of the 
tea. and that the colonists took actions to de- 
stroy the tea. No student in either group men- 
tioned during free recall that Britain was trying 
to assert its authority by maintaining the tea tax, 
or that the colonists* actions regarding the tea 
were in protest against the tax laws. Although 



the students who read the revised text did tend 
to recall that there was still a tax on tea even 
after the price was lowered, the recall protocols, 
in general, do not show major differences in the 
quality of information recalled. 

Question responses . Analyses of the ques- 
tion responses, however, did reveal some differ- 
ences in the understanding students gained from 
the two texts. These differences were particu- 
larly evident in responses to two questions re- 
garding colonial protests. The first asked why 
the colonists carried out actions such as burning 
the tea and leaving it to rot even after the price 
of tea was lowered. Twice as many students 
who read the revised text (80%) as students who 
read the original text (40%) understood that 
these actions were done in protest (e.g.. “They 
got real angry... and one night they threw it over 
the boat because they still had taxes on them.**). 
Responses from the students in the original 
group that did not mention the concept of pro- 
test (60%) showed a variety of ideas about the 
colonists' actions. These responses ranged from 
saying that the colonists were motivated solely 
by the price of tea (23%) to saying that the colo- 
nists did in fact buy the lower-priced tea (10%). 

The second question that tapped an under- 
standing of the colonists' motivations was the 
question of why the colonists threw the tea in 
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Table 6 Prototypical recall protocols for the Intolerable Acts text segments 



Original text Revised text 



It was about the Intolerable Acts . The British were 
very angry about the “Tea Party" and decided to pun- 
ish the people of Boston. The port of Boston was 
closed. And. the British made the colonists provide 
housing and food for their soldiers. Colonial leaders 
in Boston quickly sent letters to the people in the 
other colonies. People feared that their colony might 
be the next to feel the anger of Great Britain. 



iVfj/c. The italicized units in each prototypical recall are those units 



The British were very angry about the “Tea Party" 
and decided to punish the people of Boston. When 
something is intolerable it means you just can 't stand 
it. The British closed the port of Boston. That mean t 
no ships could get in or out . Food and other impor- 
tant things soon became \rry hard to get . The colo- 
nists used to be able to make some decisions on their 
own. but now Britain took those rights away. And. 
the British made the colonists provide housing and 
food for their soldiers. Colonial leaders in Boston 
quickly sent letters to the people in the other colo- 
nies. They sent help and supplies. 

that were recalled only by students who read that version of the test. 



the water. Considerably more students who read 
the revised text (66%) than students who read 
the original text (38%) understood that the colo- 
nists were trying to “get back at Britain" by 
staging the Boston Tea Party. Typical correct re- 
sponses were “They weren't going to buy it 
cause it still had tax on it and they wanted to get 
Britain mad," and “...to show the king that they 
didn't want to be bossed around." 

Another difference between the two groups 
was evidenced in their responses to questions 
regarding the agents of the action of throwing 
the tea into Boston Harbor. Both texts label the 
perpetrators of the Boston Tea Party as the Sons 
of Liberty. Neither text directly states in this 
segment that the Sons of Liberty were colonists, 
but this fact must be inferred if readers are to 
understand the implications of the events. When 
asked who the Sons of Liberty were, 60 percent 
of the students who read the revised text re- 
sponded that they were colonists, whereas only 
15 percent of the students who read the original 
text knew their identity. Similarly, when asked 
who was responsible for the Boston Tea Party, 
considerably more students who read the re- 
vised text (61 %) than students who read the 
original text (40%) were able to respond that it 
was the colonists. 

In sum, the students who read the revised 
text seemed to have a better understanding of 



die motivations underlying the colonists' actions 
and a more secure notion of the colonists' role in 
the activities portrayed in this segment of the 
text. 

The Intolerable Acts 

Prototypical recalls. Table 6 shows the pro- 
totypical recalls for the final segment of the 
text. As shown in the table, students in both 
groups typically recalled that Britain was angry 
about the Boston Tea Party and decided to pun- 
ish the people of Boston. They typically named 
two of these punishments, the closing of the 
port of Boston and the quartering of troops. In 
addition, students who read the revised text 
tended to recall the third punishment, that Brit- 
ain took away their right to self-government. 
Students who read the revised text also recalled 
the implications for the colonists of the closing 
of the port of Boston. 

The two groups also differed in their recall 
about the response of the other colonists. Those 
students who read the original text tended to re- 
call that the other colonists feared retribution, 
whereas the students w'ho read the revised text 
tended to recall that the other colonists sent help 
and supplies to the Bostonians. 

Question responses . Responses to the ques- 
tions about the Intolerable Acts suggest that the 
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students who read the revised text had access to 
some of the elements that were necessary for 
building an understanding that these acts cre- 
ated a situation of severe hardship in the Massa- 
chusetts colony. In addition, the revised group 
understood that the colonists* reactions to the 
laws promoted colonial unity, which fostered a 
Revolutionary spirit. 

One question that revealed a difference be- 
tween the groups tapped students’ understanding 
of the reason the colonists labeled Britain’s new 
laws intolerable— which was to indicate their 
shocked reaction to the harsh treatment of Bos- 
ton. Nearly half (47%) of the students who read 
the revised text understood that intolerable re- 
ferred to the British government's punishment of 
Boston. In contrast, the most common response 
from students who read the original text, given 
by nearly a third (32.5%) of the students, was 
that intolerable applied to the colonists’ action 
of wasting tea. 

Another important element in understand- 
ing the situation in Boston is knowing that the 
British, as part of their punishment of the colo- 
nists. closed the port, resulting in serious short- 
ages of food and supplies. Three questions 
focused on the closing of the port of Boston: 
What happened to the port of Boston? What 
happened when the port of Boston was closed? 
and Why did they close the port? Although most 
students in both groups replied to the first ques- 
tion that the port had been closed (revised, 
78%; original, 60%), not all students knew 
who had closed it. Of those students who re- 
sponded that the port had been closed, more 
students who read the revised text (77%) than 
students who read the original text (54%) re- 
ported correctly that it was the British who 
closed it, rather than the colonists themselves. 

There was a marked difference between the 
two groups in response to the question What 
happened when the port was closed? Of the stu- 
dents who read the revised text, 44 percent re- 
sponded that the closing of the port resulted in 
hardship for Boston (e.g. f “They were unable to 
get food and supplies”). The best that students 
who read the original text were able to do, in 
contrast, was to respond that no ships could 
come in or out of the harbor; this was the most 



common response (25%). No students in the 
original group gave any evidence of an under- 
standing that this was a hardship for Boston. 
Not surprisingly, when asked why the port was 
closed, more students who read the revised text 
(55%) than students who read the original 
(23%) realized that the port was closed in order 
to punish Boston. 

Finally, a more abstract issue tapped by the 
questions for this segment was the provision of 
the Intolerable Acts that banned colonial self- 
government. Because the principle of self-gov- 
ernment was so significant to the colonists, 
understanding Britain’s ban reinforces an under- 
standing of the severity of the hardships im- 
posed on Massachusetts. When asked about the 
meaning of the phrase No self-government was 
allowed , more students who read the revised 
text than the original text responded that it 
meant that the British took away the colonists* 
right to self-rule. Even so, only 27 percent of 
students who read the revised text and 15 per- 
cent of students who read the original text gave 
this response. Thus, it seems that an under- 
standing of this issue was beyond the grasp of 
the majority of students in both groups. 

It appears that, by the time they had com- 
pleted this final passage, most students who 
read the revised text had developed an under- 
standing of the situation in Boston and of the 
troubled relationship between Britain and the 
colonies that would serve as an appropriate base 
for understanding subsequent events. In con- 
trast, the representation that most students seem 
to have constructed from reading the original 
textbook presentation is unlikely to prepare 
them for an understanding of the outbreak of a 
revolution. 



Discussion 

The poor quality of texts is a common com- 
plaint. Thus, it may not seem surprising that 
students have difficulty understanding them or 
that better texts can be created. Our intention in 
this research was to go beyond such a straight- 
forward demonstration in order to examine the 
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quality of the improvements in comprehension 
fostered by effective revisions, and to describe 
the nature of those revisions. The effects of the 
revisions we made demonstrate that a text-proc- 
essing approach is effective for creating com- 
prehensible text. The proportion of recall and 
the proportion of questions correctly answered 
were reliably higher for the students who read 
the revised version than for the students who 
read the original text. 

The improvement in comprehension for 
readers of the revised text was not only a matter 
of remembering more of the text. The differ- 
ence between the two groups in the nature of the 
content recalled and information put forth in an- 
swers to questions most typically represented 
exactly the concepts needed to explain the 
actions of the text and to move the chain of 
events forward. For example, when recalling 
the French and Indian War segment, the readers 
of the original text tended to omit mention of the 
colonists and of who was fighting whom in the 
war. Similarly, for the section on "No taxation 
without representation," the students who read 
the revised text were more likely than the other 
students to understand that the colonists dis- 
tress over taxes arose from their desire for rep- 
resentative government. Students who read the 
original text on the Boston Tea Party were less 
li vely than those who read the revision to make 
the critical connection between the tea action 
and the tax situation. Finally, readers of the 
original text on the Intolerable Acts did not 
seem to grasp the power of the events, as evi- 
denced by their confusion over what had been 
labeled intolerable and who closed the port of 
Boston, 

There are two issues of note regarding the 
kinds of information that both groups of stu- 
dents took from the text. One is that the repre- 
sentations of the text developed by students who 
read the revised version demonstrate the suc- 
cess of our approach to revisions. Our overall 
goal was to present the events and ideas in a 
causal/explanatory style so that students would 
understand not only the events themselves but 
also why the events occurred and how events 
and ideas were related to one another. In most 
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cases, the advantage in recall and question an- 
swering of students who read the revised text 
comprised this causal-connective information. 

The other issue is the extent of confusion 
exhibited by the readers of the original text. Al- 
though we had predicted many of the problems 
students had in understanding this text, such as 
in identifying the combatants in the French and 
Indian War. comprehending the concept of rep- 
resentative government, and realizing the sever- 
ity of the Intolerable Acts, the students' lack of 
understanding of some of the more basic infor- 
mation was a bit surprising. In particular con- 
fusion about the agents who were involved in 
the actions was prevalent throughout the text. In 
reading the French and Indian War text, the stu- 
dents not only confused the agents, but they 
rarely included the colonists in their recalls of 
the passage. In reading the "No taxation..." pas- 
sage, nearly half of the students failed to realize 
that it was the colonists who were asked to pay 
taxes. In reading the Boston Tea Party text, 
many students were confused about who 
launched the Tea Party. Clearly, if confusion is 
pervasive at this level of information, little is 
going to be learned from the text. In fact, the 
representations that students develop may be in- 
accurate conceptions of the events portrayed. 

Although the revisions were successful in 
improving students' comprehension, they had 
their limitations. The study demonstrated in- 
creases in the quantity and quality of informa- 
tion recalled, but there was still room for 
improvement. One area of weakness was in the 
explanation of complex issues, such as the con- 
cept of representative government, and the Brit- 
ish act barring the colonists' rights to 
self-government. Although there were more 
students who evidenced understanding of these 
issues among the students w'ho read the revised 
text than among the students w'ho read the origi- 
nal text, the students w'ho showed improved 
comprehension still did not constitute a major- 
ity of the revised text's readers. 

The limitations of a carefully crafted revi- 
sion may at first seem rather sobering. But a 
message that one should discern from these lim- 
itations is that learning complex content is diffi- 
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cult, and simply explaining the issues better in 
the text is not likely to accomplish the whole 
task. It is important to realize that even a coher- 
ent, well-designed text is unlikely to communi- 
cate to a young learner all the information that a 
mature reader could glean from it. When learn- 
ing complex content, young students typically 
need several reiterations of ideas as well as op- 
portunities for discussion in order to clarify and 
elaborate their initial conceptions. 

Our approach to revising texts was devel- 
oped as a way to apply findings from recent 
cognitive theory and research about features 
that aid or impede reader/text interactions to the 
creation of more comprehensible texts. How- 
ever, we did not simply search for specific fea- 
tures that have been shown to be problematic, 
and execute prescribed changes at each occur- 
rence of those features. Such an algorithmic 
application of findings about reader/text inter- 
actions is not possible. For example, it is im- 
pe sible to specify outside of a specific context 
how much distance between referents can be 
tolerated before comprehension breaks down, 
how much needs to be included in a text to avoid 
knowledge gaps, or what makes for clear rela- 
tionships between ideas. 

Instead, we applied cognitive findings to 
the creation of more comprehensible texts by 
attempting to simulate the reader's mental 
processes while moving through a text. We gen- 
erated hypotheses about the reader’s text repre- 
sentation as it developed from the content 
presented, the prior knowledge required to 
understand the text at each point, and specific 
features of the text presentation. One must con- 
sider the interplay of all these factors because a 
text's content is inseparable from the effort it 
takes to make it comprehensible; the effective- 
ness of various features of texts is different for 
different content and different readers. For in- 
stance, although one of the tenets of coherence 
in discourse is explicitness of the ideas pre- 
sented, creating comprehensible text should not 
be equated with a global goal of making all con- 
cepts explicit. Decisions about which concepts 
to make explicit should be based on the role of 
each concept in a specific text situation and the 



goals of the text. 

In this study we also attempted to expose 
the reasoning underlying text revisions by de- 
scribing the operations and decisions involved 
in identifying problems and creating solutions. 
We believe this method represents an advance in 
the way revisions are described and moves be- 
yond Duffy et al.'s (1989) suggestion of creating 
a library of text problems and their solutions. 
Our approach has added, in effect, an “execu- 
tive component" that describes the thinking that 
relates the problem to the solution. We de- 
scribed how text problems were identified and 
why they might inhibit comprehension, as well 
as how we developed potential solutions and 
why the solutions might work. This approach 
addresses the concern that many researchers 
have expressed about the difficulty of describ- 
ing and communicating what goes into revi- 
sions. The approach described here might prove 
helpful to anyone responsible for creating or re- 
vising text. 

Communicating how to do something is not 
a problem in a domain that allows the straight- 
forward application of rules and procedures. 
But because of the nature of language, and the 
interactive and cumulative nature of the task of 
constructing meaning from extended discourse, 
creating comprehensible text is a very complex 
task, and its complexity precludes devising a set 
of formulaic procedures. We are therefore left 
with weak heuristics— the same heuristics that 
others have applied to text revision with some 
success, but which are not generative. What we 
have attempted to do is to anchor these heuris- 
tics by drawing connections between general 
principles— grounded in research and theory 
about the reading process — and specific text 
problems, and by describing the operations and 
decisions involved in getting from one to the 
other. In other words, we have provided models 
of text — the original and revised versions — 
along with the explanations that connect them. 
It is our notion that providing these may be 
generative because models accompanied by 
explanations can serve as mechanisms for trans- 
ferring knowledge to new instances. 
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APPENDIX 

Text and comprehension questions for the 

French and Indian Hfcr 

Original text: 

The French and Indian W hr 

In 1763 Britain and the colonies ended a 7-year war with 
the French and Indians. As a result of this war France was 
driven out of North America. Britain would now rule 
Canada and other lands that had belonged to France. This 
brought peace to the American colonies. The’ colonists no 
longer had to fear attacks from Canada. The Americans 
were happy to be a part of Britain in 1763. Yet a dozen years 
later, these same people would be fighting the British for 
independence, or freedom from Great Britain’s rule. This 
war was called the War for Independence, or the American 
Revolution. A revolution changes one type of government or 
way of thinking and replaces it with another. 

Revised text: 

The French and Indian War 

About 250 years ago. Britain and France both claimed 
to own some of the same land. here, in North America. This 
land was just west of where the 13 colonies were. In 1756. 
Britain and France went to war to see who would get control 
of this land. Because the 13 American colonies belonged to 
Britain, the colonists fought on the same side as Britain. 
Many Indians fought on the same side as France. Because 
we were fighting against the French and Indians, the war 
has come to be known as the French and Indian Mfar. 

-The war ended in 1763. Britain won the war. Now 
Britain had control of North America, including Canada. 
The French had to leave North America. 
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The colonists were .-ery glad that Britain had won. They 
now felt safer in their homes. Before the war* Indians had 
often attacked colonists who lived near the borders. Now 
Britain owned these lands where the Indians lived. The 
colonists were sure that Britain would protect them. 

The colonists were happy to be a part of Britain, but that 
was about to change. They began to decide that they would 
rather have their own country* independent from Britain. So 
a dozen years later* the colonists would be fighting the 
British for freedom from Great Britain's rule. This later war 
would be called the War for Independence* or the American 
Revolution. 



Open-ended questions: 

1. Who was fighting against whom in the war? 

2. Where did the war take place? 

j. Why was the war fought? 

4. Who won? 

5. What did they get by winning? 

6. Who had the land before? 

7. Right after the French and Indian War how wrre the 
colonists and the British gening along? 

8. At the end of the part you just read* it said something 
about what was going to happen later on. Can you 
tell me anything about that? 
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SENSE MAKING AND THE SOLUTION 
OF DIVISION PROBLEMS INVOLVING 
REMAINDERS: AN EXAMINATION OF MIDDLE 
SCHOOL STUDENTS’ SOLUTION PROCESSES 
AND THEIR INTERPRETATIONS OF SOLUTIONS 



EDWARD A. SILVER, University of Pittsburgh 
LORA J. SHAPIRO, University of Pittsburgh 
ADAM DEUTSCH, Pittsburgh Public Schools 



In this study, about 200 middle school students solved an augmented-quotient division* 
with-remainders problem, and their solution processes and interpretations were 
examined. Based on earlier research, semantic-processing models were proposed to 
explain students* success or failure in solving division-with-remainder story problems 
on the basis of the presence or absence of an adequate interpretation provided by the 
solver after obtaining a numerical solution. In this study, students’ solutions and their 
attempts and failures to “make sense” of their answers were analyzed for evidence that 
supported or refuted the hypothesized semantic-processing models. The results con- 
firmed that the models provide a solid explanation of students* failure to solve 
division-with-remainder problems in school settings. More generally, the results indi- 
cated that student performance was adversely affected by their dissociation of sense 
making from the solution of school mathematics problems and their difficulty in provid- 
ing written accounts of their mathematical thinking and reasoning. 



In recent years, research interest in children’s ability and tendency to 
“make sense” of the mathematics they learn has manifested itself in many 
different ways, such as an interest in the connections between procedural 
and conceptual knowledge (e.g., Hiebert, 1986), in the meanings children 
impose on the mathematical symbols and procedures they learn (e.g.. 
Resnick, 1988), and in the ways in which children connect or fail to connect 
school learning to everyday experience (e.g., Saxe, 1991). Interest in chil- 
dren’s meaningful interpretations of school mathematics, however, is not a 
new phenomenon; it figured prominently in the writings of Dewey (1910, 
1933), Brownell (1935, 1947) and others earlier in this century,. 



The research reported herein was supported by a grant from the Department of 
Education's Office of Educational Research and Improvement (OERI) to the Learn- 
ing Research and Development Center for the Center for the Study of Learning. The 
opinions expressed are those of the authors and should not be assumed to express the 
views of OERI. 

The authors are grateful to Mary Lynn Raith for her assistance in data collection 
and to Carol Parke for her early assistance with data analysis. Anthony Gabriele, 
Marjorie Henningsen, Patricia Ann Kenney, Margaret Schwan Smith, and several 
anonymous reviewers are also acknowledged for their helpful suggestions and com- 
ments on earlier versions of this repor*. 



132 

o 

ERIC 



129 



Volume 2, Spring 1997 



Subject-Area Learning 



Solution Processes and Their Interpretations 



This paper reports the latest in a series of studies investigating children’s 
“sense making” as it relates to the interpretation of a numerical answer 
obtained while solving division story problems involving remainders. The 
failure of American students to succeed in solving problems involving 
whole-number division with remainders has been documented through the 
National Assessment of Educational Progress (NAEP) and several state 
assessments. Only 24% of a national sample of 13-year-olds was able to 
solve correctly the following problem, which appeared on the Mathematics 
portion of the Third National Assessment of Educational Progress (NAEP, 
1983): “An army bus holds 36 soldiers. If 1,128 soldiers are being bused to 
their training site, how many buses are needed?” A similar division problem 
appeared on the 1983 version of the California Assessment Program (CAP) 
Mathematics Test for Grade 6 and was answered correctly by only about 
35% of the sixth-graders in California (Silver, 1986). In both assessments, 
students commonly erred by giving non-whole-number answers. 

To better understand the basis for the observed difficulty that students 
have in solving division problems involving remainders, several investiga- 
tions have been conducted with students in grades 6, 7, and 8 (Silver, 1986, 
1988; Silver, Mukhopadhyay, & Gabriele, 1992). The findings of these 
investigations have suggested that students’ failure to solve division prob- 
lems with remainders can be attributed, at least in part, to their failure to 
relate computational results to the situation described in the problem. 
Because the same symbolic division expression can represent different 
problem situations, the determination of the correct solution to a problem 
depends on aspects of the situational context and the quantities involved in 
the problem. Unlike the case for most other story problems encountered by 
students in elementary school, sense making is not an optional activity in 
solving these problems, because correct computation alone cannot ensure a 
successful solution. 

Recent studies (Silver, 1988; Silver et al., 1992) have examined students’ 
performance on division problem types including augmented-quotient prob- 
lems, remainder-only problems, and quotient-only problems. These studies 
examined the effects on students’ performance of their solving several divi- 
sion problems that required the same computation and similar referential 
mappings. The results indicated that students’ performance on each type of 
problem was enhanced by having them also solve related division problems. 
In general, the results were consistent with the explanation that the 
enhanced performance was due to the students’ increased sensitivity and 
attention to the relevant semantic and referential mappings involved in the 
target problem solution. In particular, experience with the related problems 
may have drawn attention to the need for mapping into either the story text 
representation or the story situation representation after obtaining a solution 
to the augmented-quotient problem through use of a mathematical model. 
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Taken together, these results and the assessment findings suggested that 
the students' failure to solve the division story problems was due, at least in 
part, to an incomplete mapping among relevant referential systems. In par- 
ticular, it appeared that students might map successfully from the problem 
text to a mathematical model (in this case, a division computation to be per- 
formed) and compute an answer within the domain of the mathematics 
model, but fail to return to the problem story text or to the story situation 
referent in order to determine the best answer to the question. Figure 1 pre- 
sents a schematic representation of a hypothesized version of a student’s 
unsuccessful solution (US) attempt. 

Story text Story situation 



From this hypothesized model, it is possible to suggest that a student’s 
successful solution attempt would be represented as follows: The solver 
maps from the story (natural language) text representation of the problems 
into a mathematical model representation, then performs the required com- 
putation within the referential system of mathematics, expressing the 
resulting answer with an appropriate mathematical representation. The 
solver then maps the computational result back either to the story text rep- 
resentation or to the implied story situation (in the “real world”) 
representation in order to decide how to treat the quotient and remainder. 
Through such a process, the successful solver would finally obtain suitable 
mathematical and natural language representations of the solution that have 
accompanying interpretations and validity within the referential systems of 
real-world situations and the knowledge domain of mathematics. Figure 2 
provides a schematic representation of the mappings involved in this ideal- 
ized successful solution (SS). 






Mathematical model 




Figure J. Schematic representation of hypothesized unsuccessful solution. 
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Until recently, evidence for these hypothesized solution models was avail- 
able only indirectly from an examination of students* performances on 
multiple-choice test items. Direct confirmation of these models of successful 
and unsuccessful solutions was obtained in an interview study (Smith & Sil- 
ver, 1991) in which the problem-solving and interpretation performance of 
eight middle school students was examined in an interview setting where the 
students solved an augmented-quotient division-with-remainders problem. 

Story text Story situation 



In addition to confirming the semantic-processing models, the interview 
protocols reported by Smith and Silver (1991) also revealed some interest- 
ing facets of students’ sense making with respect to the division problem 
task. In addition to the finding that the students who correctly solved the 
problem included a sense-making step in their solution, the interviews 
revealed that some students, who would have answered incorrectly if the 
tasks were presented in a multiple-choice format, were able to offer inter- 
esting interpretations of their numerical answers. For example, one student 
spoke of “squishing in” the extra students, and others suggested ordering 
minivans rather than a full bus for the extra students. This kind of situation- 
based thinking and reasoning remained invisible both in the multiple-choice 
response format used in the prior research and the kind of summary infor- 
mation available from the free-response solutions on the NAEF (1983) task. 

The major purpose of the study presented here was to investigate, on a 
larger scale than the Smith and Silver (1991) research, the solution process- 
es and interpretations of students solving an augmented-quotient 
division-with-remainders problem. Because the research prior to the Smith 



A 



Mathematical model 




Figure 2. Schematic representation of an idealized successful solution. 
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and Silver study had relied on multiple-choice items, little information was 
available regarding the processes used by students as they solve such prob- 
lems. In the present study, the focus was on analyzing students’ responses 
both to see if they provided evidence supporting or refuting the hypothe- 
sized referential models and to determine the extent and nature of students’ 
attempts at sense making in the context of solving a particular type of math- 
ematics problem. To facilitate the collection of more easily analyzable 
responses from a relatively large number of students, a free-response pen- 
cil-and-paper task was used. Unlike the multiple-choice tasks used in the 
prior research, this task format provides fairly direct access to students’ 
thinking and reasoning about a problem, if they are willing and able to 
express their thoughts in writing. Finally, given the evidence of situation- 
based reasoning found in the Smith and Silver study, it was hypothesized 
that the number of “leftovers” might influence students’ situation-based 
interpretations and final solutions. Thus, remainder size was chosen as a 
variable of interest. 



The sample consisted of 195 sixth, seventh, and eighth graders from a 
large urban middle school with a population approximately 40% Caucasian 
and 60% African-American of all ability levels. The students were members 
of mathematics classes taught by teachers who volunteered their classes. 

Task 

The following task was administered to each student in the sample: 

The Clearview Little League is going to a Pirates game. There are 540 
(or 532 or 554) people, including players, coaches, and parents. They will trav- 
el by bus, and each bus holds 40 people. How many buses will they need to get 
to the game? 

Three versions of this problem were used so that student responses would 
be obtained for division problems with remainder sizes equal to one-half 
(540 people), less than one-half (532 people) and greater than one-half 
(554 people). 

Administration and Procedures 

The task was administered to three classes at each grade level as a 15-20- 
minute activity during a regular class session. At each grade level, each ver- 
sion of the problem was administered to one class. The instructions 
accompanying the task directed students to show their work, to place their 
answer in an answer space provided, and to explain their answer in writing. 



METHOD 



Sample 
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RESULTS 



Since neither grade level nor ethnic differences were of particular interest in 
this study, results are reported for the aggregated sample. Students’ responses 
were examined with respect to four distinct aspects: (a) solution process, 
(b) execution of procedures, (c) numerical answer, and (d) interpretations. 
Each aspect was examined independently, and then an additional analysis was 
conducted in which various combinations of these aspects were examined. 

Solution Processes 

The solution process was defined as the set of procedures used by the stu- 
dent to obtain a numerical solution. Although there were many solution 
processes that could have been used to solve the problem, such as drawing 
pictures or forming sets, all of the students’ written responses indicated use 
of an algorithm. The majority of students used the long division algorithm 
(70%), but a significant minority (20%) of students used other algorithmic 
procedures, such as repeated addition. Table 1 exhibits the most frequently 
used algorithmic procedures and the percentage of students using each pro- 
cedure. An algorithm was considered appropriate if it could potentially lead 
to a correct solution without requiring additional procedures not in evidence 
in a student’s response. 



Table i 

Distribution of Solution Procedures 



Procedure 


Percent of student 


: usage 


Appropriate procedures 


Long division 


73% 




Repeated multiples 


7% 




Repeated addition 


5% 




Repeated subtraction 


1% 




More than one correct procedure 


5% 




Total 




91% 


Inappropriate procedures 


Addition (dividend + divisor) 


1% 




Subtraction (dividend - divisor) 


1% 




Multiplication (dividend x divisor) 


2% 




More than one incorrect procedure 


3% 




Total 




7% 


No work provided 




2% 



Execution of Procedures 



The execution of procedures referred to the actions taken by the student 
in carrying out the solution process. Since all students in the sample used an 
algorithmic procedure, examination of the execution of procedures was 
reduced to examining the correctness of the steps in the algorithm. A stu- 
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dent’s execution of procedures was judged to be correct if and only if each 
arithmetic operation in the procedure was executed without error. The work 
of those students who used an inappropriate algorithm to solve the problem 
was eliminated from examination. Table 2 shows the percentage of students 
who were successful or unsuccessful in executing the steps of the proce- 
dures for each of the appropriate algorithms used by the students. Overall, 
about 61% of the students were able to perform their calculations flawless- 
ly. The long division algorithm, though the procedure most often used by 
the students, was the most difficult for them to perform correctly. 



Table 2 



Percentage of Students with Correct Execution by Procedure 





n 


Percent with correct execution 


Appropriate procedures 
Long division 


142 


58% 


Repeated multiples 


13 


77% 


Repeated addition 


10 


80% 


Repeated subtraction 


3 


67% 


More than one correct procedure 


9 


67% 



Numerical Answer 

The number written by a student in the space provided for the answer to 
the initial problem “How many buses are needed?” was considered to be the 
student’s numerical answer. Table 3 indicates the numerical answers given 
by the students and the percentage of the sample giving each answer. 



Table 3 



Distribution of Students' Numerical Answers 



Numerical answer 


Number of students 


Percent of students 


14 


84 


43% 


13 


15 


8% 


13 and fractional remainder 


11 


6% 


13 and whole number remainder 


5 


2% 


13 and decimal remainder 


2 


1% 


1 3 and other remainder representation 


2 


1% 


Whole number remainder only 


5 


2% 


Other answers 


62 


32% 


No numerical answer 


9 


5% 


Total 


195 


100% 



Most of the response categories are self-explanatory, including those 
involving the mathematical expression of the remainder as a whole number, 
fraction, or decimal. These categories accounted for about 10% of the total 
number of student responses. 
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A few response categories deserve special attention. For example, the two 
responses in the category “13 and other remainder representation" appeared 
to involve a combination of mathematical and situation-based knowledge 
(e.g., “13 and 1 cab" and “13 and a minivan"). In the category “Other 
answers, nearly one-third of the responses involved a numerical answer 
greater than 100 — responses that resulted from students’ incorrect execution 
of the decimal division algorithm. 

Interpretations 

Interpretations were the explanations of solutions written by students in 
the space provided for that purpose. 

Coding. Interpretations were coded as “appropriate," “inappropriate," or 
no interpretation." An interpretation was coded as appropriate if, in the 
written explanation, the student said that a whole number of buses was 
needed because a fraction of a bus did not make sense or that there were 
some people who would not be able to go if an extra bus was not provided. 
In addition, if a student suggested that the fractional remainder represented 
a minivan or gave some other reasonable meaning to the numerical answer, 
the explanation was coded as an appropriate interpretation. An interpreta- 
tion was coded as inappropriate if a student explained the numerical answer 
by applying rounding or estimating rules, offered an incomplete or incorrect 
explanation, or otherwise gave evidence of confusion. A response was 
coded as involving no interpretation if it was simply an explanation of the 
procedures used to find the solution, if it was a statement commenting gen- 
erally or vaguely on the problem or the answer, or if no written explanation 
was provided. 

To ensure interrater reliability a sample of approximately one-third of the 
responses was coded independently by each of two raters. An acceptably 
high degree of agreement between raters was obtained (Kappa = .94). 

Appropriate interpretations. About one-third of the students gave 
responses that were classified as appropriate interpretations. Examples of 
interpretations considered appropriate include one student who wrote. 
You 11 need 13 and a third buses. Since buses don’t come in thirds, you get 
a whole other bus, and another student who wrote, **14 to hold everyone, 
and you would have empty seats for more people who decided to come.’’ 
Some appropriate explanations were provided for final answers other than 
14. For example, one student who gave a final answer of 13 1/2, wrote: 
520 people are riding a big bus, and you’d have to get a van for the other 
20 [people].’’ Some students who provided final answers of 13 1/2 gave 
interpretations such as, “You need 13 buses and 1 van [cab or minibus].’’ 

Inappropriate interpretations. Only 9% of the sample gave explanations 
that were classified as inappropriate. Examples of inappropriate explana- 
tions include one student who wrote: “[The answer is] 14 buses because 
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there’s leftover people and if you add a zero you will get 130 buses so you 
sort of had to estimate. Are we allowed to add zeros?” Another student, 
after attempting decimal division, reported: ‘‘[The answer is] 14. I got 
13,065 but just looking at the number I wouldn't get that so I took the first 
2 digits and added 1 because about 5 [students] would be left,” 

No interpretation. More than half of the responses for the entire sample 
were classified in this category. In addition to papers containing no written 
explanation, this category also included those papers containing only gener- 
al comments on the problem or the answer, such as the student who 
obtained an answer of 133 and wrote, “I was agassed [sic] at how many 
times it [the divisor] went in.” 

More than one-half (54%) of the sixth-grade responses were classified as 
procedural explanations (e.g., “I divided 40 into 540 and there is a remain- 
der of 20, then I reduced 20/40 to 10/20, and then I took off the zero and 
came up with 13 1/2”), and none of these sixth-grade students also provided 
an interpretation of their numerical answer. In contrast, although more than 
40% of the seventh- and eighth-grade students also gave detailed descrip- 
tions of their mathematical procedures, many also provided an 
interpretation for their solution, and their interpretations were often classi- 
fied in the other categories. In general, students' procedural explanations 
related to the execution of the steps of an algorithm or a particular represen- 
tational form of the numerical answer. 

Response Patterns 

Thus far, each aspect of a student's response has been treated indepen- 
dently. Determination of evidence that supports or refutes the hypothesized 
semantic-processing models requires examining the interplay among the 
various response components. Approximately 78% of the responses provid- 
ed direct evidence to support at least one of the hypothesized SS and US 
models, about 14% of the responses provided what may appear to be 
counter-evidence for one of the models, and the remaining 8% were judged 
to be neutral or impossible to classify with respect to the models. 

Direct supporting evidence for the hypothesized SS model was provided 
by approximately 32% of the responses. In particular, about 26% of the stu- 
dents gave a numerical answer of 14 and provided an appropriate 
interpretation, nearly 3% provided an answer other than 14 but also gave an 
appropriate interpretation, and about 3% had flaws in the execution of their 
solution procedure but were able to obtain an answer of 14 and give an 
appropriate interpretation. 

Evidence directly supporting the hypothesized US model was provided by 
approximately 46% of the responses. In particular, about 22% of the stu- 
dents correctly executed an appropriate solution procedure but provided no 
interpretation for their incorrect numerical answer; nearly 24% of the stu- 
dents incorrectly executed an appropriate procedure, gave a numerical 
answer other than 14, and provided no interpretation. 
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Possible counter-evidence for the hypothesized SS model was supplied by 
about 8% of the students, who were able to provide the correct numerical 
answer of 14 without providing any accompanying interpretation; approxi- 
mately 2%, who gave an inappropriate interpretation for the answer of 14; 
and about 4%, who gave an inappropriate interpretation for an answer other 
than 14. Further examination of many of the cases providing the apparent 
counter-evidence revealed some interesting tendencies in these responses. 
For example, a few of the students who obtained the numerical answer 14 
but provided no interpretation used a repeated addition procedure — an algo- 
rithm that is generally not associated with direct instruction in division 
story problems. In the case of the students who gave an inappropriate inter- 
pretation for the answer 14, they gave explanations involving rounding and 
estimation — topics also taught in the school mathematics curriculum. 

Remainder Size 

The written responses of the students offered no direct evidence of students 
being influenced by the size of the remainder in interpreting their solutions 
or arriving at their final numerical answer. There was, however, evidence 
that remainder size interacted somewhat with success in executing proce- 
dures and in students' tendency to interpret numerical answers. Students who 
had the problem version with remainder size equal to one-half more often 
executed their computational procedures correctly than their counterparts 
who had other versions of the problem. Moreover, students* responses for the 
problem form in which the remainder was one-half were also somewhat more 
likely to reveal some attempt to interpret the numerical answer. 

DISCUSSION 

The major goals of this study were the examination of solution processes 
and interpretations provided by students when solving an augmented-quo- 
tient division-with-remainders problem, with particular focus both on 
analyzing students* responses for evidence that supported or refuted the 
hypothesized referential mapping models proposed in earlier research (Sil- 
ver et al., 1992) and on examining the sense making evident in the students’ 
responses. A more minor goal was to determine the extent to which remain- 
der size influenced students* solutions or interpretations. 

Relation to Hypothesized Models . 

The responses provided by students in this study generally supported the 
hypothesized SS and US models. Over 70% of the students used the long 
division algorithm to solve the problem (a finding consistent with the earli- 
er NAEP report), and an additional 20% used other appropriate procedures. 
Over 60% of these students were able to execute their computation correct- 
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ly, yet only about 45% of the subjects responded with the augmented quo- 
tient (14) as their numerical answer or were able to give an appropriate 
interpretation for some answer other than 14. Overall, students’ responses 
provided considerable evidence that computational requirements were not 
the major barrier to obtaining a correct solution but rather that unsuccessful 
solutions were more often due to students’ failure to engage in interpreting 
their computational results. 

Successful solution model . Direct evidence to support the SS model is 
provided by the responses of nearly 70% of the students who gave the aug- 
mented quotient (14) as their numerical answer and also provided 
appropriate interpretations for their answer. Moreover, the few students 
who gave answers other than 14 but were able to give appropriate interpre- 
tations can also be thought of as providing “correct” solutions, and they 
provide further evidence to support the SS model. Beyond illustrating a few 
students’ attempts to relate their outside-of-school knowledge to the inter- 
pretation of a numerical solution to a mathematical problem (e.g., 
interpreting a fractional remainder as representing a minibus or van), these 
responses also illustrate a possible limitation of fixed-response formats 
(such as multiple-choice items) in assuming that only one numerical answer 
to such problems can be considered correct. 

The responses of those students who solved the problem correctly 
(obtaining an answer of 14) without making any special interpretation of the 
answer may be seen as providing possible counter-evidence for the SS 
model. Most of these students did not use long division to solve the prob- 
lem; rather they used either repeated addition, repeated subtraction, or 
repeated multiples (basically a “guess and check” strategy). These students 
executed their mathematical procedure correctly, yet they tended not to map 
back to the story situation in order to provide an interpretation. It is possi- 
ble, however, that these responses do not actually refute the SS model, 
although the responses may require some modification of a few details in 
the model. It seems reasonable to speculate that these procedures (repeated 
addition or subtraction), although more mathematically primitive than the 
long division algorithm, may be more intuitively linked to the situation 
described in the problem (filling up buses). Consider, for example, that 
adding up (or subtracting down) more naturally parallels the act of loading 
individuals on to a bus. Students who used these algorithms, unlike those 
who used long division, may have used these procedures as a natural conse- 
quence of their situation-based reasoning about the problem and may not 
have felt a need to provide an explanation of their solutions. These more 
situationally based procedures, unlike the long division algorithm, implicit- 
ly contain an interpretative framework. 

The only other possible counter-evidence to the SS model was found in the 
few responses of students who either rounded the numerical answer to get a 
whole number of buses or estimated the number of buses needed. Although 
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it could be argued that these procedures and interpretations appear to have 
arisen completely within the mathematical space of the problem and that 
they are probably not indicative of the students mapping back to the story 
text or story situation to make sense of the numerical answer, one could also 
argue that both procedures clearly reflect the situation-based constraint that 
the only allowable number of buses would be a whole number. 

Unsuccessful solution model . Students’ responses also provided consider- 
able evidence to support the hypothesized US model. More than 20% of the 
students, although able to execute correctly an appropriate computational 
procedure, provided an incorrect numerical answer and offered no interpre- 
tation of their solution. Moreover, nearly one quarter of the sample made 
computational errors that might have been corrected if the students had 
interpreted their final answers. 

Further evidence in support of the US model is provided by the responses 
of nearly 10% of the students, who solved the problem using long division 
and made a computational error involving the placement of a decimal point. 
This error resulted in students obtaining a quotient ten times as large as the 
actual numerical answer — an error that could likely have been detected if 
the students had interpreted their answer. Presumably, some students who 
used an inappropriate computational procedure in their attempted solution 
might also have detected the error if they had interpreted their solutions. 

Influence of Remainder Size 

The results of this study provided no indication that remainder size gener- 
ally appeared to influence the solution processes or interpretations provided 
by the middle school students in this sample. Nevertheless, students’ 
responses for the problem form in which the remainder was one-half were 
more likely than responses for the other forms to reveal successful execu- 
tion of computational procedures and some attempt to interpret the 
numerical answer. Further research on this issue may be needed, however, 
since the three problem forms were not equivalent with respect to computa- 
tional complexity. In order to have a problem form with a remainder of 
one-half and a divisor of 40, the dividend had necessarily to be a multiple 
of 10. This unintentional constraint resulted in the problem being an easier 
computational task than the other forms. Since the students who received 
this form outperformed the other groups in the correct execution of compu- 
tational procedures, the computational simplicity may have contributed to 
the finding that these students also more frequently engaged in interpreting 
their numerical answers than those students who solved the other forms of 
the problem. The apparent interaction between computational complexity 
and situation-based reasoning in this study is reminiscent of one of the 
major findings reported by Baranes, Perry, and Stigler (1990). 
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Additional Observations Regarding Students' Sense Making 

Form versus function. Examination of students' responses indicated that 
many students showed greater concern about the form in which their com- 
putations should be executed and in which their final numerical answer 
should be written than about the relationship between the numerical answer 
and the problem being solved. For example, many students appeared to be 
concerned about the form in which the remainder was expressed, such as 
the sixth-grade student who gave 13 34/40 as her Final answer and wrote as 
her explanation: “I got the answer and I put in [sic] a fraction because that's 
how our teacher taught us. Should I put the remainder in a fraction?" Stu- 
dents' concern with form was also evidenced in many students' detailed, 
step-by-step narrative descriptions of the procedures they used to obtain 
their numerical answers and in some students’ comments regarding what 
they believed to be the correct way to solve the problem (e. g., one seventh 
grader gave a final numerical answer of 13 and wrote, "I think dividing is 
the correct way to answer this problem.’’). 

Excessive emphasis on particular calculation procedures or notational 
forms is likely to impede students from correctly solving an augmented- 
quotient problem, especially since an interpretation of the numerical 
response is generally needed. As noted above, except possibly in the case of 
some alternative algorithms, students generally need to exit from the mathe- 
matical space in order to return to the story situation and interpret their 
numerical answer. To engage in such processing, however, a student must 
perceive the need to do so. If issues of mathematical formalism are para- 
mount in the students’ attention during problem solving, then a strong 
motivation for interpreting the numerical result is less likely to exist. It is 
encouraging to note that the older students in the sample, although placing 
considerable emphasis on matters of mathematical form like their younger 
counterparts, tended to give more consideration to the interpretation of their 
numerical results. 

The large number of responses focusing on form rather than function 
might reflect a localized instructional influence, or the Findings might sug- 
gest that middle school students are generally being taught, either explicitly 
or implicitly, that accurate computations and correct notational form are the 
most highly valued aspects of mathematics. Current middle school mathe- 
matics instruction is certainly dominated by attention to computational 
procedures, and this emphasis was reflected in the form-oriented responses 
of the students in this sample. Although it is somewhat encouraging that 
some students were able to provide interpretations and explanations that 
went beyond considerations of form, it is even more discouraging that most 
students were unable or unwilling to do so. 

Impact of task format and administration conditions. The paper-and-pen- 
cil, free-response format was intended to capture students’ solution 
processes and their situation-based thinking and reasoning. Although it 
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appeared to function fairly well as a medium for some students to communi- 
cate their thinking, it was clear that many students were neither accustomed 
to nor comfortable with explaining their thinking and reasoning. 

Some students voiced objections to having to explain their answers, and 
these came in one of two forms: (a) objections based on never having been 
taught how to explain their work and that it was a difficult thing to do, and 
(b) objections based on the apparent belief that correct computations always 
produce correct answers, thereby obviating the need for further explanation. 
Such objections (e.g., “I don’t know how to explain anything because 
there s nothing to explain. It is very hard to do this because our math 
teacher didn t teach us this.") may indicate a lack of experience in provid- 
ing written explanations for their mathematical work. 

Although the requirement of a written response in the form of an explana- 
tion or justification clearly made the task difficult for students, their 
responses appear to have been influenced by more than just that aspect of 
the task format. For example, an overwhelming majority of students pro- 
duced solutions in which the long division algorithm was used; yet there is 
anecdotal evidence to suggest that alternative algorithms may have been 
more prevalent than the students’ written work revealed. In particular, post 
hoc discussions with some of the teachers who administered the task 
revealed that some students did preliminary work on the problem on their 
desk tops or book covers. According to the teachers, many of the students 
who did this type of “scratch” work used alternative algorithms that were 
not transferred onto their papers. Instead, these students turned in to their 
teachers a final product that showed use of the long division algorithm. 
Moreover, from those teachers who engaged their classes in a follow-up 
discussion of the problem, it was learned that students not only employed a 
range of algorithms to solve the problem but also used grouping and count- 
ing techniques and drew pictures. These more mathematically “primitive” 
techniques, however, were apparently judged by the students to be unsuit- 
able for display in their written responses . 1 

The task format and administration conditions also appeared to influence 
the tendency of students to provide explanations and interpretations of 
non-whole-number answers. Anecdotal evidence from our discussions with 
teachers whose students participated in the study indicated that, during the 
discussions that followed the problem-solving activity in some classes, 
many students argued vigorously for alternative solutions using a variety of 
interpretations for the remainder and explanations of how to represent their 
interpretation numerically. Some students apparently argued that an extra 
bus was not needed because some students would be absent and would not 



These discussions were held at a summer staff development session that occurred several 
months after the task administration. Because the school year had ended and students had dis- 
persed for the summer, it was not possible to pursue these issues further in individual student 
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attend the game. Others were reported to say that some kids could walk to 
the baseball game because the school is close to the stadium (the school our 
students attended is located less than one mile from the baseball stadium 
mentioned in the problem). These rich, situation-based comments about 
possible problem solutions are reminiscent of the kinds of reasoning exhib- 
ited by some students in the Smith and Silver (1991) interview study, yet 
they were rarely found in the children’s written responses in this study. 

Children’s perception of this task as a school mathematics problem, 
rather than a task about which to be -thoughtful, may have negatively influ- 
enced their performance. The task was administered by their mathematics 
teacher as part of a regular mathematics class; hence students probably 
viewed it as a formal classroom exercise and therefore responded in a man- 
ner that they believed to be both “mathematically correct” and acceptable to 
their teacher. The formality of writing a response rather than giving it orally 
in a class discussion may also have contributed to the tendency of students 
not to reveal all of their informal thinking and reasoning about the problem. 
Although the available data make it impossible to be certain of the cause, it 
seems clear from these anecdotes that task format and administration condi- 
tions probably limited the range of students’ responses. The written, 
free-response task format was clearly useful in revealing much about stu- 
dents’ solution processes and interpretations, but these anecdotes suggest 
that the written responses may reflect only a portion of the students’ think- 
ing about the problem and that they were capable of more situation-based 
reasoning than their written responses revealed. 

Interesting confirmation for this view comes from the preliminary findings 
of Curcio and DeFranco (F. Curcio, written communication, 26 January 
1991). In their study, 20 middle school students solved two similar versions 
of the “bus” problem. The first task was presented as one of 21 interview 
items in which subjects were asked to review the work of another student 
who had solved a division-with-remainders task similar to the one in this 
study. The student’s long division calculation and an answer in which the 
remainder was expressed as a fraction were presented, and the subjects were 
asked to comment on the result and to determine what answer they would 
give for the problem. On the second task, subjects were presented with a set 
of facts involving numbers of persons going on a trip and the capacity of 
commercial vehicles available for transportation. The students x were asked to 
make a telephone call (on a teletrainer) to order transportation for a school 
trip. Students had far greater success in solving the second task (17 out of 20 
correct) than the first task (12 out of 20 correct). On the second task, the stu- 
dents in the Curcio and DeFranco investigation were more likely to apply 
situation-based reasoning and interpretation. For example, six students 
ordered a number of vans and then asked for “like a car or something” to take 
the remaining students. Another student argued that some students would 
surely be absent, so it would be unnecessary to order an extra van. Although 
the findings reported by Curcio and DeFranco are intriguing, design limita- 
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tions in their study prevent a clear determination of whether student respons- 
es are due to the effects of problem context or problem order. Nevertheless, 
the findings generally suggest that the performance context can be a powerful 
influence on students’ problem-solving processes and reasoning. 

From the discussion above, it appears that the general performance con- 
text in this study, that is, the combination of task format and administration 
conditions, may have led students to respond in ways that emphasized 
behaviors they believed to be mathematically relevant, such as the applica- 
tion of formal algorithms, and give little attention to the exhibition of 
behaviors they believed to be mathematically irrelevant, such as situation- 
based reasoning and interpretations. Surely, these issues related to the 
influence of context on performance need to be considered in future assess- 
ments of students’ problem solving and sense making. 

CONCLUSIONS 

The findings of this study contribute to our understanding of student per- 
formance in solving division-with-remainder problems and contribute more 
generally to our understanding of the relationship between situation-based 
sense making and mathematical problem solving. Semantic-processing 
models, incorporating Kintsch’s (1986) important distinction between story 
text and story situation, were proposed to explain students’ success or fail- 
ure in solving division-with-remainder story problems on the basis of the 
presence or absence of an adequate interpretation after the solver obtains a 
numerical solution. The results obtained from the paper-and-pencil instru- 
ment administered in this study confirmed that the models provide a solid 
explanation of students’ oft-reported failure to solve such problems in 
school settings, especially for students using the long division computation- 
al procedure. In fact, it is likely that the models could be extended to 
explain students' success or failure in the solution of multistep story prob- 
lems in which the result of one calculation step must be interpreted prior to 
the execution of a second calculation step. Multistep problems are notori- 
ously difficult for students, and a simple extension of the US model 
proposed in this study may provide a semantic-processing explanation for 
the difficulty that goes beyond other explanations based primarily on the 
syntactic features of such problems (Goldin & McClintock, 1979). 

Ajthough the findings of this investigation demonstrate the power of the 
proposed models in identifying a semantic-processing deficiency associated 
with unsuccessful solution, the findings also illustrate the limitations of the 
models in accounting for two important mediational factors: the school per- 
formance context and the difficulty of providing a written explanation. The 
anecdotal reports of students engaging in more situation-based reasoning 
than was evident in their written responses suggest that students’ failure to 
provide an appropriate interpretation may be due, at least in part, to their 
dissociation of school mathematics performance from more natural ways of 
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thinking and reasoning about the world. Viewed from this perspective, it is 
clear that a general model to explain students’ successful or unsuccessful 
solution of such problems would need to consider the performance context 
and its associated constraints. 

The suggestion that student performance may have been adversely affect- 
ed by the dissociation of sense making from the solution of school 
mathematics problems points to the need for more instructional attention to 
sense making as a part of school mathematics instruction. Yet, even if 
mathematics instruction were to make stronger connections between school 
mathematics problems and procedures and the forms of thinking and rea- 
soning prevalent in out-of-school situations, the results of this investigation 
suggest that other issues must also be addressed in order to improve stu- 
dent performance. 

As noted above, students had great difficulty in providing written expla- 
nations of their reasoning. Some students may have been more capable of 
explaining their thinking and reasoning orally than they were able to do in 
writing, and this points to the need for written explanations to become a 
more prevalent feature of school mathematics instruction. Assignments 
requiring students to provide explanations for their work are consistent with 
calls for greater emphasis on communication in the mathematics classroom 
(NCTM, 1989; Silver, Kilpatrick, & Schlesinger, 1990). The results of this 
study suggest that, unless and until solution explanations and interpretations 
become a regular item on the menu of instructional activities in mathemat- 
ics classrooms, it is unlikely that many students will spontaneously engage 
in such activity when it is appropriate to do so. Moreover, until students are 
more accustomed to explaining their mathematical thinking and reasoning 
in writing, researchers will be stymied in their efforts to gather, in an effi- 
cient manner on a large scale, rich descriptive data about student thinking. 

Since the task used in this study may have appeared to students as being 
primarily about solving a problem rather than interpreting a solution, a 
potentially fruitful area for further investigation involves the examination 
of alternative assessment settings or formats that might target more directly 
the need for sense making and solution interpretation. For example, the type 
of written task used by Curcio and DeFranco, in which students were asked 
to examine the work of another student and determine or critique a pro- 
posed answer, might be used to stimulate students’ situation-based 
reasoning, because the computational burdens would be removed and the 
goal-directedness toward problem solution might be reduced in favor of 
interpretation and sense making. 

The findings regarding students’ use of alternative algorithms and the 
likely connection between these alternative procedures and situation-based 
reasoning about the problem suggests another interesting area for future 
investigation. In their study of problem solving across different contexts, 
Baranes et al. (1990) reported an apparent relationship between students’ 
use of solution strategies and certain contextual features of the problem sit- 
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uation. Although limited in scope, the findings of Baranes et al., taken 
together with the results of the study reported here, suggest the potential 
value of examining more closely the relationship between problem-solving 
procedures and situational contexts. 

In addition to extensions of the proposed semantic-processing models to 
incorporate important mediational factors influencing students* likelihood 
of engaging in solution interpretation, and in addition to continued research 
related to the general issue of understanding how and when students con- 
nect mathematics to situations, the findings of this study suggest the 
wisdom of developing and implementing instructional activities in which 
children are challenged to engage in mathematical sense making. Clearly, 
this study has shown that American students in the middle grades need 
much more experience in explaining their solutions to mathematical prob- 
lems. If students continue to dissociate thoughtfulness from the solution of 
problems, there can be little hope of substantially improving the poor math- 
ematics performance of American students. 
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